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FOREWORD 


The  Human  Factors  Technical  Area  of  the  Army  Research  Institute  (ART)  is 
concerned  Kith  human  resource  demands  of  Increasingly  complex  battlefield 
systems  used  to  acquire,  transmit,  process,  disseminate,  and  utilize  informa¬ 
tion.  This  increased  complexity  places  great  demands  upon  the  operator  inter¬ 
acting  with  the  machine  system.  Research  In  this  area  focuses  on  human 
performance  problems  related  to  interactions  within  command  and  control  centers 
as  well  as  Issues  of  system  development.  The  research  program  includes  both, 
technology  base  and  advanced  development  research  as  well  as  a  limited  amount 
of  technical  advisory  service  (TAS)  to  Army  agencies  and  activities.  The 
general  purpose  of  TAS  is  to  provide  immediate  consulting  asslstarce  in  meeting 
short-term  priority  requitanents. 

One  area  of  special  Interest  Involves  the  development  of  estimates  for 
the  contributions  of  human  factors  in  military  systm  development.  The 
inquiry  into  the  topic  resulted  from  a  tri-service  committee  decision  to 
investigate  the  possibility  of  providing  system  designers/managers  with 
^Idence  of  the  value  of  human  factors  to  compare  with  other  pertinent 
information  from  engineers,  operations  research  analysts  and  system  analysts. 
This  initial  report  emphasizes  the  methodological  considerations  of  such  an 
undertaking  and  creates  a  foundation  for  Implementation  of  such  an  effort 
by  system  personnel. 

The  following  individuals  contributed  to  this  effort:  Dr.  Edgar  M. 

Johnson  and  Dr.  Thomas  M.  Granda  (ARI)  ;  Mr.  Paul  Linton  (Naval  Air  Development 
Center)  ;  Mr.  John  L.  Miles,  Jr.  (Human  Engineering  Laboratory)  ;  Dr.  Donald 
A.  Topmiller  (USAF  Aerospace  Medical  Research  Laboratory) ;  and  Dr.  Alfred 
R.  Fregly  (Air  Force  Office  of  Scientific  Research). 


THE  CONTRIBUTION  OF  HUMAN  FACTORS  IN  MILITARY  SYSTEM  DEVELOPMENT; 
METHODOLOGICAL  CONSIDERATIONS. 

BRIEF 


Requirexnent : 

To  determine  the  conceptual  basis  for  humam  factors  contri¬ 
butions  to  military  system  acquisition  and  development.  Given 
this  conceptual  basis,  to  determine  a  feasible  method  for 
evaluating  the  contribution  of  human  factors. 

Procedure ; 

Two  parallel  analytic  processes  were  used  to  determine  a 
conceptual  basis  and  feasible  methodology  for  assessing  the 
contribution  of  human  factors  in  system  development. 

•  A  first  analytic  process  involved  the  development  of 
a  rationale  for  human  factors  in  system  development, 
followed  by  a  determination  of  the  existing  basis  for 
humem  factors  R&O  (ranging  from  formal  DOD  requirements 
to  informal  documentation) .  This  culminated  iu  a 
determination  of  the  conceptual  basis  for  identifying 
human  factors,  contributions,  through  emalysis  of  human 
factors  principal  products,  system-specific  efforts, 
and  technology  base. 

•  A  second  analytic  process  was  undertaken  to  detepnihe 

a  feasible  method  for  evaluating  human  factors  contribu¬ 
tions,  including  the  identification  of  metrics  for 
measuring  the  value  of  human  factors .  Concurrent  with 
this,  a  review  of  cost- benefit  analysis  techniques 
was  conducted.  Out  of  these  efforts,  an  impact  assess- 
.  ment  methodology  emerged  as  the  most  feasible  methodology 
for  measuring  the  value  of  human  factors  contributions 
in  military  system  development. 

■  ‘  ■  .  .  .  .  ■  .  _ : _ _ i-'-  .r  -  '  - 


Product: 


Several  items  worthy  of  note  include:  the  development  of  a 
conceptual  basis  in  which  specific  human  factors  efforts  and 
products  for  each  phase  of  system  development  were  defined;  a 
preliminary  set  of  measurement  metrics  were  developed;  the 
framework  of  an  impact  assessment  methodology  for  evaluation  of 
human  factors  R&D  was  developed;  and  an  impact  assessment 
vocabulary  hieraurchy,  tailored  to  human  factors,  was  specified 
(i.e.,  impact  areas,  metrics,  and  empirical  measures).. 

Utilization: 

The  conceptual  basis  for  human  factors,  in  conjunction  with 
the  impact  assessment  methodology,  can  be  used  to  advantage  by 
practitioners  who  wish  to  determine  the  contributions  of  human 
factors  R&D  to  military  system  development.  Although  the 
methodology  requires  additional  refinement,  and  validation  through 
case  examples,  the  present  approach  can  be  implemented. 
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CHAPTER  1 
INTRODUCTION 

Project  Objectives 

In  a  sense,  the  present  project  represents  an  attempt  to 
confront  an  irony:  while  the  sophistication  of  military  systems 
is  increasing,  the  attention  paid  to  the  capabilities  and  limi¬ 
tations  of  those  people  who  must  operate  and  maintain  them  has 
not  been  increasing.  The  arena  in  which  this  irony  can  be 
most  effectively  confronted  is  the  process  of  military  system 
development. 

The  overriding  objective  of  the  present  project  is  to 
enhance  the  value  of  all  O.S.  military  systems.  The  route  to 
that  objective  that  is  paramount  for  the  present  effort  is  the 
assurance  that  the  human  factors  contribution  to  military  system 
development  is  timely,  of  an  appropriate  quantity,  and,  most 
explicitly,  that  it  is  of  high  quality. 

In  another  sense,  then,  we  are  concerned,  with  the  quality 
control  of  what  is  widely  regarded  as  an  essential  ingredient 
in  the  overall  military  system  development  effort.  If  we  follow 
our  own  precepts,  however,  we  know  that  quality  assurance  depends 
on  accurate  feedback  and  that  feedback,  in  turn,  depends  on 
evaluation. 

Consequently,  the  key  ingredient  in  achieving  the  broader 
objective  is  the  establishment  of  the  means  to  evaluate  the 
contribution  of  human  factors  to  military  system  development. 

It  is  the  construction  of  that  key  ingredient  that  has  been  the 
immediate  objective  of  the  project. 
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Tne  Boundaries  of  huiaon  Factors 


In  order  to  identify  and  measure  the  contribution  of  human 
factors  it  is  necessary  to  define  human  factors.  It  is  also 
appropriate  not  only  to  provide  a  definition  but  also  to  provide 
some  classification  of  human  factors  RaD  and  a  discussion  of  its 
scope,  for  reasons  that  should  become  clear  shortly.  Collectively 
these  terms  will  be  called  the  "Boundaries  of  Human  Factors 
Research  and  Development. "  Each  of  these  boundaries  will  be 
briefly  described  below. 

Definition  of  Human  Factors 

A  comprehensive  definition  of  iiuman  factors  R&D  that  is  used 
by  DOD  and  all  services  does  not  exist.  This  is  not  to  suggest 
that  wc  do  not  already  know  essentially  what  human  factors  is, 
but  rather  to  suggest  that  we  are  not  interested  in  a  precise, 
academic  definition  of  human  factors  for  this  study. 

For  the  purposes  of  this  report,  we  know  that  human  factors 
is  one  of  the  four  categories  of  people- related  research  funded 
as  part  of  the  RDT&E  budget  of  the  DOD.  These  four  categories 
have  been. defined  by  the  Military  Assistant  for  Training  and 
Personnel  Technology  in  the  pfiice  of  the  Under  Secretary  of 
Defense  for,  Research  and  Engineering  in  some  recent  briefing 
materials.  These  definitions  are  provided  in  Exhibit  1-1. 

Another  DOD  definition  of  human  factors  is  contained  in  the 
Technology  Coordination  Paper  for  FY  1978  (Department  of  Defense, 
1979): 

Hum^m  factors  technology  is  concerned  with  the 
design,  development,  evaluation,  and  deployment 
of  manned  systems  so  that  human  operators  would 
be  able  to  operate  and  maintain  military  systems 
at  their  optimum  performance  level.  This  includes 
the  systematic  investigation  of  how  the  design  of 
a  person*  s  job  and  the  tools  that  are  provided 
affect  his  capacity  to  do  a  job. 


1-2 


Exhibit  I'l 
DOD  Definitions* 


•  HUMAN  FACTORS- 

Dtw«topm«nt  of  improvtd  methods  and  technologies  for  the  analysis,  design, 
and  evaluation  of  oquipment/tvttems  for  safer  and  more  efficient  operation 
and  maintar\ar>ce. 

•  PERSONNELSiMANPOWER- 

Developmant  of  tachniques/ntethods  for  irtilizing  available  personnel  resources 
through  improved  selection,  job  assignment,  organizational  analysis,  and 
management  techniques  to  meet  combat  available  and  projected  force  needs. 

•  EDUCATION  &  TRAINING - 

Development  of  aducational/training  methods  and  media  for  managing, 
designing,  and  evaluating  new  gerteration  instructional  systems  for  military 
applications. 

•  SIMULATION  &  TRAINING  DEVICES- 

Oevtlopment  of  cost-effective  training  equipment  and  technology  that 
produce  the  rreeded  performance  for  operation  and  maintenance  of  military 

systems. 


*Thif  chart  is  from  a  brief  provided  by  the  Military  Assistant  for  Training  and  Personnel  Technology 
(OUSORAE). 

This  DOD  definition  perhaps  defines  the  boundaries  of  human 
factors  in  terms  of  its  technical,  domain;  but  metrics  for  deter¬ 
mining  the  value  of  human  factors  and  costs  for  assessing  the 
affordability  of  human  factors  need  to  be  more  adequately  defined. 
Conceirning  these  two  points,  a  few  statements  can  be  made  that 
are  useful  in  shaping  this  important  definition; 

.  •  Metrics  for  measuring  the  value  of  human  factors  must 

i  include  measures  of  both  system  capability  and  cost. 

We  will  also  define  the  area  of  man-system  compatibility 
as  a  category  of  metrics.  .  Further,  human  factors  efforts 
on  products  must  also  relate  to  system  performance. 
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•  Affordability  of  human  factors  must  be  assessed  not  only 
in  terms  of  dollars  spent  but  also  in  terms  of  cost 
avoidance  (through  reduced  selection,  manpower,  or 
training  requirements) . 

•  User  acceptance  must  be  part  of  the  value  of  human  factors. 
Changes  in  personnel  attitude  not  only, contribute  to 

more  effective  use  of  the  equipment  or  system,  but  may 
have  long-term  effects  on  issues  such  as  attrition  and 
retention. 

In  summary,  there  seems  to  be  a  consensus  that  human  factors 
includes  effective  integration  of  man's  role  and  performance  into 
system  operation  and  maintenance. 

Classification  of  Human  Factors  RDT&E 

Another  boundary  of  h'oman  factors  that  is  important  is  the 
classification  of  the  work.  Again,  DOD  has  standardized  this 
dimension  for  RDT&E.  Exhibit  1-2  shows  the  three  classifications 
for  human  factors  work  and  gives  some  indication  of  what  is; 
included  in  each  class  (Fiorello  et  al. ,  1979).  '  A  cursory 
examination  of  these  classes  suggests  that  they  may  correspond 
roughly  to  categories  of  R&D  funding  (6.1,  6.2,  and  6.3),  at 
least  with  respect  to  the  technology  base,  but  this  possibility 
must  be  explored  in  more  detail  (see  Chapter  4). 

Scope  of  Human  Factors 

Perhaps  the  most  limiting  boundary  of  human  factors  has  been 
the  scope  of  its  integration  into  the  system  development  process. 

In  brief,  there  has  been  precious  little  utilization  of  human 
factors  in  the  early  phases  of  system  acquisition,  particularly 
in  the  Mission  Analysis  and  Concept  Development  Phases.  There 
has  been  more  utilization  in  the  DemonstrationAalidation  Phase; 
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and  by  far  the  greatest  utiliiatioi;  occurred  in  the  Full- 

Scale  Development  Phase,  where  the  traditional  human  engineering 
or  man-machine  interface  design  occurs*  perhaps'  the  reason  for 
the  lack  of  human  factors  in  the  earlier  piases  of  system  devel¬ 
opment  is  the  lack  of  recognition  tiiat  toere  is  bot’i  a  product 
and  a  payoff  to  be  had  during  these  .early  j  hases. 

Kxliibil  1  tl 

Cla^<^ificatioll  of  llinudii  Fa>  loi>  Kli  IXK 


•  Human  Rtlattd  Studitt 

*What  are  tha  eapabilitiat  and  limitationt  o(  uperatcr^  and  mamtaineri  lynems/tubtyrtems’" 
Emphatit  it  on  incraating  our  itate  of  knowladge  concerning  Sumans'  operational  performance 
Included  ere  deu,  performartce  methods  leievant  to  ph'  tical  diaracterittict.  tentory  and  motor  cape 
bilitiat,  and  human  information  procauing 

•  Human-Machina  Ratattd  Studies 

"How  do  we  allocate  fi.rsctions  between  people  and  equipment^”  This  is  often  termed  "lubtystem" 
related  because  it  oorscerns  the  design  of  a  tptcific  man-machine  relationship  Included  are  etfons 
dealing  with  computer-aided  methods  for  human  anginaering.  workload  maasuremant  techniques, 
designing  for  maintainability',  control  and  display  dasign,  and  workspace  layout. 

•  Human-Machine-Miuion  Related  Studies 

"How  are  total  configurations  of  people  end  equipnitni  conttru.-ted  for  maximum  tactical  and 
strattgic  effactivanats^"  This  concarns  the  optimum  combination  of  individual  and  taam  performance 
within  the  total  operational  system.  This  combination  applies  not  only  to  major  ground,  sea.  air 
and  subsurface  synems.  but  to  the  command  and  control  of  these  systems  as  well 


The  principal  products  from  each  phase  of  systein  acquisition 
should  be  a  meaningful  way  to,  represent  the  scope  of  human  factors 
in  a  military  system  development.  These  phase-products  should 
vary  in  specificity  from  the  very  conceptual  requirement  level  to 
the  very  detailed  design  level,  just  as  is  the  case  with  products 
of  engineering  logistics,  etc.,  during  each  phase.  After  a  great 
deal  of  analysis  and  synthesis  (discussed  primarily  in  Chapters  2, 
3,  and  4),  a  set  of  human  factors  products  has  been  defined.  For 
the  purposes  of  this  report,  the  principal  human  factors  products 
from  each  major  phase  of  the  system  acquisition  process  are 
identified  below. 


1-5 


MAJOR  PHASE 

OF  SYSTEM  ACQUISITION 

Mission  Analysis  Phase 

Concept  Development  Phase  —  Allocation  of  System  Functions  to  Man  as  a 

part  of  the  Decision  Coordinating  Paper  (DCP) 

Demonstration/Validation  -  Task  Analysis  and  Determination  of  Human 

Phase  Engineering  Requirements 

Full-Scale  Development  Phase  —  Design  of  the  Optimal  Man-Machine  interfaces 

A  more  definitive  explanation  of  these  products  is  offered 
in  Appendix  A.  Additionally,  the  payoff  or  value  that  can  be 
realized  at  each  phase  of  system  acquisition  will  be  explored. 

For  the  moment,  the  purpose  of  identifying  the  principal  products 
is  to  bound  the  scope  of  what  we  mean  by  human  factors,  and  to 
suggest  that  each  product  will  require  some  costs  and  must  yield 
some  benefits. 


PRINCIPAL  HF  R&D  PRODUCT 

—  Development  of  the  Role  of  Man  as  a  part  of  a 
Mission  Element  Needs  Statement  (MENS) 


The  Integration  of  Human  Factors  and 
Military  System  Development 

The  problem  of  proper  integration  of  the  human  factors 
conLribution  into  the  development  of  military  systems  can  be 
drzunatized  by  asking  the  question,  "V?hat  result  would  occur 
if  the  development  process  involved  zero  human  factors 
participation?"  A  completely  non- controversial  answer  to  that 
question  can  be  cast  in  a  statistical  framework.  That  is,  you 
would  get  a  distribution  of  outcomes.  In  fact,  the  best  hypo¬ 
thesis  would  be  that  the  distribution  would  be  symmetrical  and 
near-Gaussian  because  of  the  multitude  of  influences  at  work. 
Such  a  hypothetical  distribution  is  presented  in  Exhibit  1-3 
as  the  dashed-line  cur^ra.  The  solid-line  curve  represents  the 
characteristics  of  the  distribution  shift  (again  hypothetical) 
when  the  human  factorc  contribution  is  introduced  early  and 
continuously  throughout  the  development  process.  (One  should 
probably  interpolate  a  family  of  distributions  to  represent 
various  degrees  of  human  factors  participation  at  less  than 
optimum  levels. ) 

Exhibit  1-3 

Hypothetical  Distributibn  of  System  Development  Outcomes 
With  and  Without  Optimum  Human  Factors  Participation 
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Caref\.l  eoceunination  of  the  distributional  joodel  presented 
in  Exhibit  1-3  can  help  in  the  process  of  vmderstcuiding  vaurious 
aspects  of  the  resistemce  to  human  factors  pcurticipation  on  the 
part  of  some  managers  of  militaury  system  development  efforts. 

It  is  clear  from  the  diagram,  for  example,  that  some  systems 
could  achieve  acceptance  (in  the  sense  of  going  into  full-scale 
production)  with  little  or  no  formal  human  factors  peurticipation. 
Thus,  if  the  program  manager  were  "lucky,”  he  could  avoid  the 
cost  of  that  contribution.  In  the  same  vein,  there  are  some 
systems  below  the  rejection  line  that  did  have  optimum  humem 
factors  participation — substantiating  the  point  that  such 
participation  does  not  guaremtee  success. 

However,  the  diagram  also  reveals  that  a  rational  strategy 
would  be  one  which  always  incorporated  human  factors  participation 
The  program  manager  who  does  otherwise  is  simply  playing  against 
the  odds  if  the  postulated  relationships  in  Exhibit  1-3  are  valid. 


Overview  of  the  Project 


The  present  project  constitutes  very  much  of  a  team  effort. 
At  the  top,  the  planning,  analysis,  and  review  process  was 
carefully  coordinated  among  representatives  of  the  three  services 
the  COTR,  and  the  contractor.  The  project  was  scheduled  into 
three  phases  which  were  intentionally  set  up  to  be  partially 
iterative,  but  with  a  definite  pattern  of  progression.  At  the 
completion  of  each  phase,  a  comprehensive  review  was  made  .and 
highly  specific  feedback  and  orientation  was  provided  by  the 
review  panel  to  the  contractor.  Between  formal  review  sessions, 
guidance  was  provided  by  the  COTR  and  other  advisors  from  the 
HFE-TAG. 

A  useful  overview  of  the  component  structure  of  the  project 
is  provided  by  Exhibit  1-4.  To  a  degree,  each  box  corresponds 
to  a  major  effort  of  the  project,  but  does  not  correspond  to  a 
chapter  in  the  report.  Those  boxes  on  the  left  side  of  the  chart 
represent  the  efforts  to  identify  the  contribution  of  human 
factors  in  system  development.  The  results  of  these  efforts 
are  reported  in  Chapters  2,  3,  and  4.  Those  boxes  on  the  right 
side  of  the  chart  represent  the  efforts  to  identify  a  framework 
for  evaluating  human  factors  contributions.  The  results  of 
these  efforts  are  reported  in  Chapters  5  and  6,  The  final  box 
corresponds  to  the  final  chapter  in  the  report. 

The  main  message  that  should  be  drawn  from  Exhibit  1-4  is 
the  intricacy  of  the  relationship  between  the  efforts  and  the 
essential  symmetry  of  that  relationship.  These  attributes  are, 
at  the  same  time,  both  the  cause  and  the  result  of  the  teamwork 
within  the  working  level  of  the  project..  Human  factors  content, 
human  factors  research  methods,  military  system  development 
procedures,  program  management  practices,  and  the  philosophy 


Exhibit  14 

Relatioruhip  of  the  Major  Efforts  of  the  Study 


and  methcxiolcgy  of  cost-benefit  analysis  and  related  approaches 
all  had  to  be  brought  into  the  work  effort  and  made  to  be 
mutually  constructive.  The  diversity  is  represented  in  the 
diagram.  The  constructive  aspect  is  symbolized  by  the  symmetry 
of  the  links. 

Specific  ex2unples  of  particular  applications  of  human 
factors  to  particuleu:  systems  and  instances  of  various  attempts 
to  evaluate  the  impact  of  such  applications  are  scattered 
throughout  the  narrative,  but  excunples  of  the  linkage  between 
human  factors  products  and  impact  analysis  criteria  as  mediated 
by  system  metrics  are  also  contained  in  an  appendix  (Appendix  C) . 
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Human  Factors  in  Military  Systems 

In  Chapter  2,  the  question  of  the  contribution  of  human 
factors  to  the  development  of  military  systems  is  considered 
in  historical  context.  The  evaluation  of  the  speciality  is 
shown  to  have  been  based  on  tangible  results.  It  is  also 
revealed  that  while  the  content  of  the  contribution  was  often 
based  on  quite  rigorous  scientific  procedures,  the  evaluation 
of  the  contribution,  in  the  sense  of  establishing  human  factors 
as  em  essential  ingredient  in  military  system  development,  was 
either  neglected  or  handled  by  anecdotal  evidence. 

The  anecdotal  evidence  is  persuasive  as-  far  as  it  goes. 
Moreover,  the  philosophical  base  upon  which  both  humain  factors 
specialists  and  system  engineers  and  the  other  disciplines  were 
working  tended  to  be  an  increasingly  integrated  one  during  the 
period  from  the  late  1940s  to  the  late  1960s. 

During  this  same  period,  the  formalization  of  the  role  of 
human  factors  in  military  system  development  was  accomplished. 
The  first  generation  of  official  directives  has  since  been 
revised  many  times,  but  the  most  recent  versions  contain  clear 
reiterations  of  the  recognition  of  the  requirement  for  contri¬ 
butions  from  human  factors  sources  that  was  first  officially 
articulated  in  the  1950s.  A  summary  of  the  formal  basis  for 
humem  factors  participation  in  military  system  development 
is  presented  in  the  first  section  of  Chapter  3.  The  formal 
instigations  are  seen  to  be  linked  to  the  structured  chronology 
of  the  system  development  process.  This  very  structure  sets 
the  stage  for  the  further  conceptualization  of  the  mode  of 
contribution  that  is  the  subject  of  the  balance  of  Chapter  3. 

In  these  sections,  the  key  message  is  that,  the  contribution  of 
human  factors  can  be  characterized  as  specific  products,  each 


of  which  is  tied  to  a  stage  in  military  system  development. 
Further,  a  simple  model  has  been  prepared  which  identifies 
specific  hiiman  factors  efforts  that  make  up  the  products  of 
each  system  development  phase.  Finally,  in  Chapter  4,  there  is 
a  brief  treatment  of  how  human  factors  R&E  from  the  technology 
base  can  also  contribute  to  the  principal  human  factors  products 
of  each  system  development  phase. 

This  conceptualization  of  human  factors  contributions 
reinstates  the  condition  of  the  tcingible  consequence  that 
characterized  the  early  development  of  the  speciality.  As  such, 
the  stage  is  fvirther  set  for  the  possibility  of  a  rigorous  evalu¬ 
ation  approach,  one  that  represents  a  clear . advemce  over  the 
retrospective  and  amecdotal  approaches  of  the  past. 

The  Measurement  of 
Human  Factors  Contributions 

The'  concern  for  a  feasible  method  to  measure  and  evaluate 
the  human  factors  contribution  to  military  system  development  did 
not  suddenly  arise  as  part  of  an  attempt  to  rescue  human  factors 
from  oblivion.  The  concern  is  primarily  a  manifestation  of  two 
parallel  trends  in  the  much  larger  arena  of  public  administration 
These  trends  can  be  briefly  characterized  as  a.  growing  sense  of 
a  need  for  stricter  accoiintability  in  the  expenditure  of  govern¬ 
mental  resources  emd  the  evolution  of  rationalistic  procedures 
for  providing  such  accountability. 

Military  systems , development  is  unquestionedaly  an  area  of 
expenditure  of  government  resources.  The  major  public  policy 
issues  that  are  associated  with  these  processes  have  to  do  with 
allocation-of- resources  decisions.  Should  public  funds  be 
invested  in  system  X  or  system  Y  or  system  Z?  Such  allocation 
decisions  presumably  should  be  guided  by  2malytic  results  in 
the  form  of  estimations  of  relative  returns  on  investment  (ROD . 


Below  the  level  of  choice  between  X,  Y,  and  Z  are  a  series 
of  subordinate  choices.  Having  decided  on  X  as  the  best  poten¬ 
tial  system,  what  should  be  the  proportional  investment  in 
technology  A,  B,  and  C?  When  either  A,  B,  or  C  might  meUce  a 
crucial  contribution  to  the  effectiveness  of  system  X,  what  mix 
will  yield  the  best  potential  payoff? 

At  the  most  basic  level/  these  questions  are  technical 
questions  for  the  political  economist.  And,  indeed,  it  has 
been  from  that  source  and  from  disciplines  such  as  Operations 
Research  and  others  that  have  an  intellectual  affiliation  with 
political  economics  that  the  evolution  of  rationalistic  proce¬ 
dures  has  come.  A  central  contribution  of  this  evolutionary 
effort  has  been  the  formal  methodologies  under  the  rubric  of 
cost-benefit  analysis.  Consequently,  the  cost-benefit  approach 
is  used  as  a  starting  point  for  the  specific  tailoring  of  a 
methodology  to  meet  the  objectives  of  the  present  project. 

As  it  turns  out,  the  strict  monetary  criteria  required  in 
formal  cost-benefit  anailysis  mcUce  it  awkward  to  apply  ih  its  pure 
form  to  our  central  problem.  However,  the  basic  logic  of  the 
methodology  and  its  inherent  emphasis  on  quantification  do  pro¬ 
vide  a  productive  orientation.  This  orientation  is  reflected  in 
Chapters  5  and  6,  which  cover  measurement  metrics  aaid  methodology, 
respectively. 

The  point  is  that  the  work  reported  here  is  well  within  a 
conceptual  movement  that  might,  by  this  time,  be  called  a  tradition 

Of  specific  precedents,  however,  there  have  been  precious 
few.  There  are  so  few,  in  fact,  that  to  achieve  some  perspective 
on  the  present  enterprise,  it  is  necessary  to  examine  another 
parallel  literature:  that  of  evaluation  in  education  and  training. 
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The  specific  application  of  approaches  resembling  those  of 
cost-benefit  analysis  had  its  own  broad  history  in  the  general 
movement  toward  treating  education  in  a  more  scientific  way  (see, 
e.g.,  Campbell  and  Stanley,  1963).  The  lift-off  point  came, 
however,  in  the  m.id-to-late  1960s  with  the  work  of  Suchman  (1967) 
and,  more  particularly,  Stufflebeam  (1968).  The  past  decade  has 
witnessed  the  production  of  many  hundreds  of  articles  and  reports, 
most  of  which  were  focused  on  the  actual  evaluation  of  some  par¬ 
ticular  educational  or  training  program  or  a  particular  training 
technology,  and  a  few  of  which  were  focused  on  methodological 
advances,  as  such,  or  what  might  be  called  the  "management  of 
evaluation."  An  up-to-date  example  of  the  former  is  provided  by 
the  recent  works  of  Orlansky  and  String  (1977  and  1979)  in  their 
summary  evaluations  of  the  cost-effectiveness  of  flight  simulators 
and  computer-based  instruction  in  military  training.  An  example 
of  the  latter  mode  is  provided  by  the  work  of  Conley  et  al. 

(1979),  in  their  review  of  six  major  government-sponsored  training 
programs.  Conley  and  his  group,  who  work  for  the  U.S.  Office  of 
Personnel  Management  (formerly  the  Civil  Service  Commission), 
were  concerned  with  the  very  issues  central  to  this  effort:  the 
need  for  evaluation,  its  feasibility,  and  the  procedures  for 
doing  it  in  such  a  way  that  the  outcome  can  be  used  to  guide 
management  decisions. 

The  key  point  is  that,  agdin,  a  s^stemtial  precedent  exists 
for  the  general  approach  and,  in  these  instances,  the  approach 
has  been  more  or  less  successfully  applied  to  an  activity  that 
is  considered  to  be  somewhat  "soft"  in  the  sense  of  its  being 
exclusive  of  rigorous,  quantitative  measurement. 

To  return  to  the  precious  few  direct  emtecedents ,  three 
exeunples  can  be  cited.  Each  is  of  quite  a  different  sort. 


One  such  case  is  the  work  of  Geer  (1979).  In  a  sense,  it 
is  the  more  remote  precedent  because  the  document  as  a  whole 
is  devoted  to  the  problem  of  how  to  conduct  human  engineering 
analyses.  Only  a  few  pages  (out  of  220)  are  devoted  to  the 
evaluation  of  the  human  engineering  contribution  (pp.  24-25). 

The  approach  is  built  on  a  brief  review  of  the  TMI-2  incident 
and  is  affirmative  euid  non-quantitative  in  character. 

A  second  case  is  represented  by  the  work  of  Price  et  al. 
(1979).  In  this  case,  evaluation  is  the  central  objective  but 
the  substance  to  be  evaluated  is  the  human  factors  effort  in 
research  rather  than  scientific  system  development.  The  approach 
taken,  however,  is  to  relate  research  (technology  bas;e)  outputs 
to  such  achievements  as  cost  avoidance  in  both  military  operations 
and  training.  In  short,  the  benefit  side  of  the  cost-benefit 
ratio  is  stressed. 

The  third  case  is  represented  by  a  report  compiled  by  the 
BDM  Corporation  (1980)  that  focuses  on  the  human  factors  aspects 
of  aircraft  accidents.  As  such,  the  focus  is  much  narrower  than 
that  of  the  present  effort.  However,  it  is  closely  akin  from  a 
methodological  point  of  view,  in  that  an  attempt  is  made  to 
evaluate  the  human  factors  contribution  to  aircraft  accident 
prevention  using  an  analysis  of  return  on  investment. 

In  a  sense,  then,  we  have  come  full  circle. .  The  present 
work,  particularly  as  reflected  by  the  contents  of  Chapters  5 
and  6,  represents  an  extension  and  focusing  from  three  sources:  . 
the  general  source  of  allocation  analysis  in  the  public  policy 
domain  typified  by  the  methodology  of  formal  cost-benefit 
analysis;  the  parallel  source  exemplified  by  attempts  to  apply 
such  rigorous  methods  to  the  evaluation  of  the  elusive  processes 
of  education  and  training;  and  the  very  restricted  source  of 
specific  attempts  to  evaluate  human  factors  contributions  through 
a  linkage  to  some  aspect  of  system  effectiveness. 
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Chapter  5  reveals  how  the  focus  can  be  realized  by  means 
of  qiiantitative  measures  that  fit  into  the  broader  frarowork 
of  system  engineering.  Chapter  6  describes  how  the  measurement 
operations  can  be  made  and  how  they  can  be  interpreted  through 
the  use  of  a  conceptual  model  adapted  from  the  basic  structure  , 
of  formal  cost-benefit  analysis. 

The  Concept  of  Impact  Areas 

A  specific  case  is  made  in  Chapter  6  that  strict  cost- 
benefit  procedures  are  not  appropriate  for  the  evaluation  of 
the  contribution  of  human  factors  to  military  system  development. 
The  central  reason  is  that  strict  cost-benefit  models  require  a 
single  ultimate  criterion:  monetary  value.  It  turns  out  to  be 
not  only  awkward  but  occasionally  ludicrous  to  reduce  the  human 
factor  aspect. to  a  dollar  measure. 

Consequently,  a  compromise  was  sought.  The  goal  bec^une 
that  of  deriving  a  methodology  that  would  be  as  close  to  strict 
cost-benefit  methods  as  possible  while  covering  the  full  scope 
of  the  human  factors  contribution.  The  existing  derivation  of 
cost-benefit  methods  that  met  that  goal  was  policy/impact 
assessment. 

The  adoption  of  impact  assessment  as  an  exemplary  procedure 
opened  the  door  to  another  crucial  adaptation.  That  is,  it  was 
discerned  that  the  scope  issue  could  best  be  met  by  adding  a 
criterion  factor  called  compatibility  to  the  basic  two,  already 
labeled  cost  and  capability. 

Thus,  a  triad  of  criteria  were  adopted:  cost,  capability, 
and  compatibility,  and  the  members  of  the  triad  were  designated 
as  impact  areas  to  directly  connote  that  the  proposed  methodology 
was  to  be  a  version  of  policy/ Impact  assessment. 


The  cost  criterion  is  used  in  essentially  the  same  way  as  it 
is  in  the  cost-benefit  methodology.  It  is  expressed  in  dollars 
and  pertaxns  to  the  total  life  cycle  costs  of  a  system. 

The  capcdjility  criterion  is  very  close  to  the  benefit  com¬ 
ponent  of  the  cost-benefit  -Tiethodology  except  that  it  pertains 
ultimately  to  system-mission  performance  and  is  not  reduced  to 
dollar  value  in  the  impact  assessment  version  of  the  methodology. 

The  compatibility  criterion  is  uniquely  responsive  to  the 
substance  of  the  human  factors  contribution.  As  a  concept  in 
its  own  right,  it  links  logically  to  the  consensual  goal  of 
human  engineering,  which  is  to  achieve  an  optimum  match  among 
human,  machine,  and  mission. 

Because  compatibility  is  something  of  an  innovation  in  the 
lexicon  of  evaluation  methodologies,  it  seems  useful  to  give  it 
a  little  extra  attention.  Specifically,  we  can  break  it  down 
into  its  constituent  parts:  physical  compatibility,  physio¬ 
logical  compatibility,  and  psychological  compatibility.  Physical 
compatibility  refers  to  the  human  as  a  physical  object  having 
certain  dimensions  of  size,  weight,  reach,  etc.  design  of 

workplaces  such  as  the  cockpit  of  an  aircraft  must  provide  for 
these  physical  attributes. 

Physiological  compatibility  refers  to  the  human  as  a  func¬ 
tioning  orgemism.  Thus,  meted^olic  processes  such  as  respiration 

must  be  taken  into  account  in  design.  Also,  factors  such  as 

» 

visual  acuity  under  differing  conditions  of  illumination  are 
physiologically  based  and  the  designer  errs  if  he  or  she  specifies 
a  display  that  cannot  be  read  under  operational  conditions. 


Psychological  compatibility  is  the  most  complicated  of  the 
constituents.  It  breedcs  down  further  into  behavioral  and  atti- 
tudinal  components.  ,  Behaviors  relate  primarily  to  established 
habits  such  as  the  hedait  of  turning  a  knob  clockwise  to  increase 
some  effect  cmd  counterclockwise  to  decrease.  Design  should  be 
responsive,  to  such  habits. 

The  attitudinal  component  is  m2mifest  mainly  in  the 
phenomenon  of  user  acceptance.  Even  under  strict  military 
discipline,  users  can  reject  involvement  with  a  particular 
system.  The  reasons  might  not  be  entirely  logical  but  can  be 
nonetheless  powerful.  The  prediction  of  such  reactions  or  even 
their  measurement  after-the-fact  goes  well  beyond  conventional 
engineering  considerations  but  lies  well  within  the  province 
of  human  engineering.  It  is  in  this  domain  that  the  need  for 
special  observational  methods  and  measurement  techniques  becomes 
most  obvious  and  where  some  of  the  particular  justifications  for 
adaptations  or  extensions  of  strict  cost-benefit  models  derive. 

It  is  reconciling  these  variants  with  the  mainstream  of  system 
engineering — bringing  these  aspects  back  into  the  family,  so  to 
speak--that  constitutes  one  of  the  major  contributions  of  the 
present  project. 

Summary  and  Synthesis 

The  total  effort  involved  in  the  present  project  has  already 
been  revealed  to  be  complex  in  the  sense  of  being  a  composite  of 
several  different  topics  and.  orientations.  This  point  is  made 
even  more  dramatically  in  Exhibit  1-5,  which  represents  an  attempt 
to  characterize  the  intended  outcome  in  a  composite  summazy  format 

The  central  core  of  the  representation  is  the  system  devel¬ 
opment  process  itself.  It  is  shown  as  consisting  of  four  phases 
denoted  by  Roman  numerals. 
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The  lower  portion  of  the  diagram  stresses  the  role  of  the 
technology  as  provider  of  the  problem-solving  resources.  In 
particular,  the  engineering  disciplines  are  specified,  including 
humem  factors. 

In  a  similar  way,  the  role  of  technology  base  activities  is 
shown:  to  support  the  evolution  of  the  problem-solving  resources 

The  upper  part  of  the  diagram  emphasizes  the  feedback  mech¬ 
anism  introduced  at  the  beginning  of  this  chapter.  The  function 
of  the  impact  analysis,  in  the  first  instance,  is  to  provide 
techniques  for  data  gathering.  The  convention  used  is  intended 
to  show  that  different  techniques  can  be  used  under  different 
circumstances  and  that  scnne  choice  must  be  made. 

The  outcome  of  intact  assessment  is  then  shown  to  feed  back 
primarily  to  the  problem-solving  resources  box  because  that  box 
is  also  the  locus  of  the  management  decisions  in  military  system 
development.  That  box,  indeed,  is  the  ultimate  target  of  the 
project  reported  here. 
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CHAPTER  2 


A  RATIONALE  FOR  THE  VALUE  OF  HUMAN  FACTORS 
IN  SYSTEM  DEVELOPMENT 


The  substance  of  this  chapter  includes  a  brief  review  of  the 
historical  evolution  of  human  factors  in  military  system  develop¬ 
ment  and  a  compilation  of  the  established  arguments  in  favor  of 
the  utilization  of  the  human  factors  resource  by  program  managers 

One  conclusion  is  that  those  involved  in  one  or  another 
aspect  of  military  system  development— particularly  during  the 
past  ten  years  or  so— have  allowed  what  should  be  a  strong 
collaboration- type  relationship  to  take  on  some  of  the  features 
of  an  adversary  relationship.  The  present  project  constitutes 
an  attempt  at  "rapprochement"  by  expanding  upon  the  established 
rationales  for  the  use  of  human  factors  resources  through  the 
elucidation  of  methods  permitting  a  mors  explicitly  objective 
assessment  of  their  value. 

For  those  readers  who  are  already  convinced  of  the  value  of 
human  factors  in  military  system  development,  this  chapter  can 
serve  only  as  a  quick  refresher  course  with  some  special  emphasis 
on  the  constraints  involved  in  achieving  timely  participation. 

For  the  more  skeptical,  the  chapter  should  reveal-  that  the  effort 
to  pronote  human  factors  in  military  system  development  does 
indeed  have  an  established  rationale  that  is  substantial,  even 
though  its  powers  of  persuasion  have  not  been  overwhelming  in 
recent  times. 
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Historical  Background  of 
Military  Human  Factors 

The  history  of  human  factors  in  the  military  has  been 
reviewed  many  times  (see  Meister  &  Rabideau,  1965;  Christensen, 
1976;  and  Chaikin,  1978)  and  will  not  be  dwelt  upon  here. 
Essentially,  there  is  a  consensus  that  the  major  requirement 
for  human  factors  contributions  to  system  design  occurred  during 
World  War  II  and  grew  out  of  earlier  work  in  aercmedical  research, 
industrial  psychology,  and  industrial  engineering.  As  expressed 
by  Meister  and  Rabideau  (1965) ; 

With  World  War  II  ,a  new  factor  entered  which  had 
tremendous  consequences  for  human  factors.  This  was 
a  period  of  increases  in  technological  complexity, 
involving  such  new  systems  as  radar  euid  sonar  and 
highly  complicated  fighter  and  bomber  aircraft, 
designed  to  be  used  in  new  environments  and  under 
highly  demanding  conditions.  These  conditions,  under 
which  the  operator  could  not  function  as  readily  as 
he  had  before,  complicated  medical,  physiological, 
and  psychological  requirements  for  design. 

In  the  lore  of  human  factors,  the  oral  tradition  asserts 
that  in  the  early  stages  of  mobilization,  the  Army  in  particular 
had  a  surplus  of  psychologists.  Fpr  lack  of  a  better , assignment , 
two  or  three  of  these  individuals  were  given  the  "detail"  of 
reviewing  a  rash  of  P-47  accidents.  Several  of  these  aircraft 
had  crashed  when  the  flaps  had  been  lowered  on  takeoff  when  the 
wheels  were  still  down.  The  problem  turned  out  to  be  a  confusion 
of  two  controls  (flaps  and  wheel  retraction)  by  pilots  who  had 
been  trained  on  an  aircraft  in  which  the  flap  and  wheel  controls 
were  reversed  from  their  location  in  the  P-47  cockpit,  the 
ability  of  the  psychologists  to  "solve"  the  problem  convinced 
some  key  officials  that  it  would  be  a  good  idea  for  a  psychologist 
to  look  at  all  new  systems  to  ensure  that  no  "traps"  were  included 
in  the  design  for  the  unwary  operator. 
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Later,  when  the  major  electronic  systems  were  being  brought 
into  operation,  human  factors  participation  had  become  more  or 
less  routine.  The  so-called  “team  approach"  that  included  the 
human  factors  scientist  had  become  a  basic  rule  in  the  minds  of 
such  system  pioneers  as  Churchman,  Ackoff,  Arnoff,  Roy,  and  Flood. 

It  is  notable  that  at  this  still-early  stage  in  the  history 
of  the  speciality  one  generic  problem  was  characterized  as  system 
COTiplexity.  We  were  already  seeing  systems  with  large  numbers  of 
interactive  components  where  the  functional  relationship  between 
such  components  was  not  self-evident  to  ordinary  operators  or 
maintenance  technicians  and  where  many  of  the  components  were  not 
"familiar"  to  such  personnel.  The  current  situation  with  respect 
to  military  systems  is  a  direct , extension  and  major  expansion  of 
the  trend  that  started  at  that  time.  Forty  years  later  we  are 
still  talking  about  the  complexity  of  military  systems  and  their 
demands  on  the  capabilities  and  limitations  of  people. 

No  more  history  need  be  provided  here  except  to  make  two 
further  points.  The  first  is  that  during  the  years  from  World 
War  II  until  today,  an  enormous  amount  of  human  factors  data  has  ■ 
been  developed,  much  of  which  has  been  inco::porated  into  hand¬ 
books,  guidelines,  specifications,  standard: 

An  excellent  overview  of  this  historical  de’ 
by  Chaikin  (1978)  ,  who,  also  charts  the  deve, 

1472,  Human  Engineering  Design  Criteria  for 
Equipment,  and  Facilities  (see  Exhibit, 2-1) 

*Along[  these  lines ,  it  might  be  useful  to  c( 
that  the  human  factors  vocation  is  to  some 
own  successes.  In  a  sense,  the  early  find: 
of  human  factors  specialists  have  become  a 
practice.  If  the  technology  had  remained  i 
only  very  slowly,  the  whole  story  would  ha^ 
condition  in  which  the  human  factors  speci, 
himself  out  of  a  job." 


!5,  directives,  etc. 
velopment  is  provided 
■  Lopment  of  MIL-STD- 
Military  Systems, 

,*  The  second  point 


jnsider  the  possibility 
extent  a  victim  of  its 
Lngs  (and  admonitions) 
standard  engineering 
:he  same  or  had  evolved 
re  concluded  with  a 
ilist  has  '^worked 
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Exhibit  2-1 

Development  of  MlL-STD-1472 


is  that  despite  this  enormous  accumulation  of  data  and  the 
directives,  standards,  specifications,  etc.  for  enforcing  the 
application  of  human  factors  in  military  systems,  that  portion 
of  hiiman  errors  which  generally  can  be  attributed  to  deficiencies 
in  equipment  system  design  is  still  a  severe  problem  in  the  Army, 
Navy,  and  Air  Force  today. 

The  issue  of  complexity  and  the  problem  of  human  error  and 
its  relationship  to  system  effectiveness  will  be  discussed  in 
the  next  two  sections.  A  separate  section  on  optimal  manned 
systems  and  the  compatibility  factor  will  follow.  The  final 
three  sections  will  ad4ress  the  questions  of  opportunity,  using 
what  we  know,  and  the  affordability  of  human  factors. 
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Military  System  Complexity 
and  Human  Factors 


The  notion  that  the  complexity  of  military  systems  was 
the  driving  force  behind  human  factors  (or  human  engineering) 
literally  coming  to  life  in  World  War  II  is  still  with  us. 

For  30  or  40  years  someone  has  been  saying  that  the  current  or 
about- to-be- introduced  systems  are  complex  and  require  special 
consideration  of  human  elements.  During  this  same  time  period, 
however,  no  one  seems  to  have  defined  complexity,  nor  to  have 
quantified  it.  What  is  apparent  is  that  systems  are  at  least 
different  today  from  what  they  were  30  years  ago,  and  that  most 
systems  in  the  military  are  being  rapidly  replaced  by, newer, 
high-technology  versions.  In  any  event,  the  complexity  theme  is 
appropriate  to  establish  the  value  of  human  factors  in  militaiy 
systems  development. 


Complexity  (or  some  synonym)  has  been  mentioned  at  all 
levels  of  the  Department  of  Defense  recently.  For  example. 
Dr.  Harold  Brown,  the  Secretary  of  Defense,  in  his  annual 
report  to  the  Congress,  FY  79,  stated  the  following  (pg.  9); 

.  .  .  Modernization,  in  some  cases,  has  brought 
with  it  shorter  mean- times  to  failure,  longer  repair 
times,  and  increased  training  requirements,  as  well 
as  greater  sophistication  and  capability  of  equipment. 
Inflation,  increased  pay,  and  the  need  to  modernize 
our  forces  have  meant  curtailed  funds  for  operation 
and  maintenance. 

.  .  .  Accordingly,  we  must  keep  up  our  training  not 
only  because  U.S.  forces  may  be  sent  into  action  with 
very  little  advance  warning,  but  also  because  we  rely 
increasingly  on  the  sophistication  of  our  equipment 
to  compensate  for  potential  superiority  in  enemy 
numbers.  It  is  equally  essential  that  our  war  reserve 
stocks  be  maintained,  mostly  for  our  own  needs,  but 
to  some  degree  for  Asian  allies  as  well.  At  the  same 
time,  we  must  raise  the  percentage  of  our  equipment 
that  is  combat-ready  because,  owing  to  unit  costs, 
we  have  less  of  it  to  bring  to  bear  in  an  emergency. 


2-6 


1 


To  put  the  matter  bluntly,  unless  we  are  prepared 
to  maintain  these  components  of  readiness,  collec¬ 
tive  security  and  deterrence  will  be  seriously 
undermined  .... 

Dr.  Brown  refers  to  the  sophistication  of  modern  equipment 
rather  than  complexity.  He  also  talks  about  the  need  to  raise 
the . percentage  of  equipment  that  is  combat-ready;  and  while  he 
does  not  say  so  in  the  excerpt,  all  military  system  readiness 
is  measured  in  terms  of  both  equipment  readiness  and  personnel 
readiness.  Obviously,  a  piece  of  equipment  that  can  perform  its 
mission,  but  that  wilt  not  because  the  operator  cannot  operate 
or  maintain  it,  is  not  combat-ready. 


General  William  E.  DuPuy,  in  a  speech  in  1977  describing 
the  Army  training  system,  also  deals  with  the  complexity  issue. 
General  DuPuy  talks  about  problems  converging  on  the  Army  and 
makes  the  following  statements: 

The  first  thing  that  is  converging  is  all  that  new 
equipment.  The  rate  of  introduction  of  new  equipment 
will  increase  exponentially.  The  first  thing  that 
goes  like  that  is  the  amount  of  equipment  that  is 
going  to  arrive  in  the  Army  between  1978  and  1985. 

And  the  Army  has  to  digest  it.  Traditionally  armies 
have  a  hard  time  digesting  new  things.  We  all  do, 
especially  organizations  like  armies.  Anyway,  that's 
the  first  area  of,  convergency.  You  are  going  to  be 
innudated  with  new  tanks,  new  MICV's  [Mechanized 
Infantry  Combat  Vehicles],  new  TACFIRE's  [Tactical 
_ Direction  Systems],  Battery  Computer  Systems, 

ROLANDS,  a  whole  new  set  of  communications 
equipment,  a  whole  new  set  of  electronic  warfare 
equipment,  and  on  and  on  .  .  . 

Complexity  is  another  problem  converging  on  the  Army 
Every  single  new  system  being  fielded  is  more  complex 
than  the  one  it  replaces.  This  complexity  is  getting 
to  be  more  of  a  problem  than  just  operating  and  main¬ 
taining  it.  But  the  complexity  of  this  new  set  of 
equipment  raises,  if  you  will,  integrated  complexity. 


The  issues  of  introduction  of  new  equipment  and  complexity 
can  be  dramatized  by  examining  Exhibits  2-2  and  2-3.  These 
exhibits  are  from  recent  briefings  on  the  human  factors  program 
at  the  Naval  Air  Development  Center.  Exhibit  2-2  indicates  the 
dual  problem  of  increasing  information  requirements  for  aircraft  ' 
operators  and  the  decreasing  available  cockpit  space  for  providing 
displays  or  controls.  As  may  be  seen,  the  last  weapon  system  on 
that  chart,  the  AV-8  (Harrier)  V/STOL  aircraft,  has  approximately 
one-third  of  the  cockpit  space  that  the  F-4  aircraft  has. 

Exhibit  2-2 

'  Cockpit  Space  and  Information  Requirements 
for  Several  Navy  Aircraft  Weapon  Systems* 

Available  G>ckpit  Space  (in^) 


•Tliia  chart  )*  from  a  briaf  prevWad  by  tha  Human  Factor*  Cnginaarlnf  OlvWon  of  a«*  Naval  Air  Oavalopmaot 
Canlar,  Warmlnttar,  Panntylvanlai 
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Exhibit  2-3 

Trend  of  Accident  Rates  for  Typical  Navy  Aircraft  and  the  AV-8A  Harrier* 


1971  IS?**  1973  1974  1975  1976  1977 


FISCAL  YEAR 

*Thit  chart  it  from  ■  brief  provided  by  the  Humen  Fectort  Engineering  Oivition  of  the  Navel  Air  Davalapment  Center, 
Warmintter',  Penniylvania. 

Exhibit  2-3  shows  a  disconcerting  trend,  neunely,  that  the 
current  V/STOL  accident  rate  is  increasing.  This  is  contrary  to 
the  experience  typically  encountered  when  new  aircraft  are  intro¬ 
duced.  Furthermore,  "pilot  factor"  as  a  contributing  cause  seems 
alarmingly  high.  With  respect  to  this  last  point,  the  data  shown 
in  Exhibit  2-3  represent  21  accidents,  16  of  which  occurred  in 
the  V/STOL  flight  regime  (l.e..  Conversion  flight,  landing,  or 
takeoff).  Of  these  16,  11  had  pilot  factor  as  a  contributing 
cause.  It  should  also  be  added  that  the, Naval  Air  Development 
Center  has  since  initiated  a  program  to  provide  human  factors 
support  early  in  the  design  of  V/STOL  aircraft. 


Human  Error,  Human  Factors, 
and  System  Effectiveness 


The  operators  and  maintainers  of  military  systems  do  make 
errors  and  many,  if  not  most,  of  these  errors  can  be  traced  to 
faulty  design.  In  a  classical  study  by  Shapero  et  al,  (1960) , 
a  survey  of  nine  Air  Force  missile  systems  showed  that  human 
error  contributed  from  20-53%  of  system  unreliability.  These 
percentages  referred  only  to  human  errors  during  field  exercises 
with  these  systems,  i.e.,  errors  during  the  launch  or  relaunch 
activities.  The  study  did  not  attempt  to  go  back  into  the  life 
history  of  each  system  and  find  human  errors  in  the  design  and 
fabrication  of  each.  Swain  (1964)  comments  on  the  Shapero  report 
in  the  following  way: 

Human  errors  have  a  greater  effect  on  system  relia¬ 
bility  than  many  people  realize  ....  In  most 
cases,  it  is  more  efficient  to  redesign  procedures 
and  equipment  which  can  minimize  or  even  eliminate 
certain  types  of  human  errors.  Engineers  can  be 
taught  valuable  design  principles  to  minimize  human 
error,  but  engineers  cannot  take  the  place  of  a 
human  factors  specialist. 

Swain  also  emph^asizes  the  human  engineering  problem  based  on  a 
Sandia  investigation  of  human  error  which  analyzed  a  large  number 
of  production  defects  at  the  plant  of  an  AEC  prime  contractor. 

It  was  found  that  82%  of  the  defects  could  be  directly  attributed 
to  human  error. 


Meister  and  Rabideau  (1965)  also  discuss  the  problem  of 
human  error  and  develop  the  link  between  human  error  and  human 
factors  and  system  effectiveness.  They  quote  some  human  error 
percentages  (page  15)  from  other  sources  which  estimate: 

...  that  40%  of  the  problems  uncovered  in  missile 
testing  derive  from  the  human  element.  63.6%  of  the 
(shipboard)  collisions,  flooding  and  grounding  could 
be  blamed  3n  hiunan  error.  Reports  produced  by  the 
United  States  Air  Force  indicate  that  human  etror 
was  responsible  for  234  of  313  aircraft  accidents 
during  1971. 
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More  recently,  Coburn  (1973) ,  in  a  paragraph  describing 
the  benefits  frcxn  systematic  human  engineering,  again  states 
the  problem  of  human  error  and  its  relationship  to  human  factors 
(human  engineering) : 

The  payoff  in  conducting  a  systematic  human 
engineering  program  is  realized  in  improved  system 
performance,  reduced  training  cost,  improved  mcinpower.  ' 
utilization,  fewer  errors  and  accidents,  reduced 
maintenance  costs,  higher  probedsility  of  mission 
success,  and  improved  user  acceptance.  Without 
applying  a  systematic  humeui  engineering  program, 
attainment  of  an  effective  ship  system  is  fortuitous 
and  improbable. 

Failure  to  apply  systematic  human  engineering  can  be 
costly — research  indicates  that  typically  up  to  40% 
of  all  ship  system  malfunctions  are  attributable  to 
human  error.*  Even  increasing  automation  cf  ship 
systems  does  not  eliminate  the  application  of  human 
engineering  programs,  since  mam  is  still  involved  as 
a  user  and  maintainer. 

To  maximize  the  payoffs  previously  cited,  human 
engineering  must  be  applied  throughout  the  ship 
system  life  cycle.  It  starts  with  inputs  to  planning 
documents  and  continues  throughout  concept  formulation, 
contract  definition,  engineering  development  and 
production,  test  and  evaluation,  and  f inally , fleet 
operations. 


*PicJcrel,  E.W. ,  fc  McDonald,  T.A.  Quantification 
of  human  performance  in  large  complex  systems. 

Human  Factors,  1964,  6,  647-662. 

Finally,  the  effect  of  human  error  was  dramatically  illustrated 
recently  in  Tims  magazine  (January  8,  1979)  by  the  photographs 
sho%#n  in  Exhibit  2-4  and  the  simple  description  noting  human  error, 
which  is  included  here  verbatim. 


Exhibit  2-4 


A  Trident  Missile  Teat  at  Cape  Canaveral 
Photographs  from  TIME.  January  8, 1979  by  Mark/Norman  Suminey 


A  MISSILE'S  UPS  AND  DOWNS 


It  was  one  of  the  most  dramatic  flights  in  missile 
history,  all  recorded  in  these  .exclusive  photographs 
for  TIME.  The  Trident  had  hardly  left  it6  launching 
pad  at  Cape  Canaveral  when  it  started  to  wobble 
wildly.  About  500  ft.  in  the  air,  it  suddenly  made 
^  boomerang  turn;  then  exploded  and.  smashed  to  earth 
125  yds.  from  its  takeoff  site.  The  fallen  missile 
burned  fiercely  for  10  min.,  sending  a  column  of 
white  smoke  soaring  skyward,  but  no  one  was  injured: 
It  was  the  third  failure  out  of  17  Tridents  tested 
to  date,  and  the  cause  was  human  error.  Someone 
included  an  extra  step  in  the  check  list,  which  led 
to  the  guidance  system's  shutting  down  a  half-second 
before  ignition.  The  missile,  which  is  designed  for 
submarine  launching,  was  out  of  control  from  the 
moment  of  takeoff.  The  Navy's  calm  term  for  the 
Trident's  destruction:  "No  test." 


\ 

\ 


2-12 


The  point  o£  the  above  discussion  on  human  error  is  that 
the  real  value  of  human  factors  is  to  reduce  actual  or  potential 
human  error  or  increase  human  reliability.  It  is  not  just  human 
relicibility  but  the  consequences  of  human  reliability  on  system 
relicdsility  that  are  important.  The  relative  impact  of  human 
error  on  system  reli2U3ility  has  been  shown  graphically  by  Meister 
auid  R2d}ideau  (1965) .  They  discuss  this  graph  (Exhibit  2-5)  in 
their  own  words  as  follows: 

.  .  .  shows  the  relationship  between  human  relia¬ 
bility  Rh  and  equipment  relicibility  Rg  and  their 
contribution  to  overall  system  reliability  Rg. 

Thus,  an  Rg  of  .85  coupled  with  an  Rjj  of  .90 
produces  an  Rc  of  approximately,  .78.  Lower  the 
human  reliability  to  .30  with  the  same  equipment 
reli2d}ility  of  .85,  and  system  reliability  now 
becomes  .25.  It  is  apparent,  therefore,  that 
anything  which  decreases  Rg  must  be  a  primary 
concern  of  the  human  engineer.  It  is  assumed 
that  much  of  this  error  results  from  inadequacies 
in  system  design  which  create,  favorable  conditions 
for  error  occurrence. 

Error  potentiality  resulting  from  inadequate  design 
can  only  be  eliminated  by  systematic  and  continuing 
evaluation  through  the  development  of  that  design. 

If  the  human  factors  aspects  of  system  performance 
are  not  routinely  evaluated,  it  is  very  likely 
that  they  will  be  overlooked,  with  the  result  that 
the  particular  system  function  involved  will  be 
developed  inefficiently. 


Exhibit  2-5 


Effect  of  Human  and  Equipment  Reliability 
on  System  Reliability,  Rg  x  R||  =  Rg 


mmmu 
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All  human  error  is  not,  of  course,  a  function  of  poor  human 
factors.  In  a  paper  included  in  the  Minutes  of  the  12th  Training 
and  Personnel  Technology  Conference  on  "Human  Factors  in^ Weapon 
System  Test  and  Evaluation"  (Taylor,  1978),  John  Miles  of  the 
U.S.  Army  Human  Engineering  Laboratory  developed  a  chart  of 
various  sources  for  attribution  of  human  error.  This  chart  has 
been  modified  and  expanded  and  is  included  as  Exhibit  2-6.  As 
can  be  seen,  there  are  five  basic  sources  which  can  account  for 
human  error  in  military  systems;  and  there  are  five  types  of 
deficiehcies  in  equipment  system  design  which  human  factors 
application  should  resolve  or  minimize.  It  ia  also  worthy  of 
note  that  the  five  typical  sources  of  human  error  (the  top  of 
the  chart)  may  also  be  turned  around  and  viewed  as  the  five 
principal  techniques  for  achieving  performance  competency  in 


Exhibit  2-6 

Typical  Sources  for  Attribution  of  Human. Error 


personnel.  These  five  techniques  for  performance  achievement  are 
discusse<^  by  Price  (1979)  .  He  points  out  that  human  engineering 
(human  factors)  is  seldom  used  early  in  system  design  to  obtain 
performance  competency.  Traditionally,  performance  competency 
is  obtained  through  selection  and  training;  and,  more  recently, 
emphasis  is  being  placed  on  improved  job  performance  aids.  Still 
more  recently,  motivation  is  receiving  more  emphasis.  All  of 
these  approaches  have  their  place  and  their  problems.  In  brief: 

•  Training  is  becoming  extremely  expensive  and  is  "under 
the  gun"  by  DOD  and  Congress. 

•  Job  performance  aids  are  showing  great  promise,  but  have 
yet  to  prove  their  cost  effectiveness  on  a  long-term 
basis. 

•  The  ability  to  select  for  aptitude  is  not  a  course  that 
is  readily  available  in  the  military  any  more,  as  is 
pointed  out  by  Rook  (1965) : 

The  more  rational  course  is  to  use  the  capa¬ 
bilities  of  people  as  we  find  them  and  to 
create  situations  in  which  the  job  at  hand 
can  be  done  by  the  people  we  get,  rather  than 
only  by  the  people  we  wish  we  had. 

'•  Finally,  motivation  as  a  way  of  reducing  human  error  is 
important;  and  it  is  also  important  for  the  long-term 
effects  that  can  be  achieved  through  job  design  (see 
Price,  1979)  .which  affect  the  total  personnel  system. 
However,  from  a  performance  point  of  view.  Rook  has 
pointed  out  that  "motivational  schemes  have  nearly  always 
produced  transient  results  in  which  maximum  performance 
increases  are  usually  about  30%  or  less."  Rook  also 
discusses  that  a  much  greater  potential  for  reducing 
error  is  by  modifying  the  performance  situation  (human 
factors).  He  states  that: 


The  aiQount  which  errors  can  be  reduced  and 
quality  improved  by  changing  environmental 
conditions  is  virtually  unlimited — if  your 
money  emd  time  are  also  unlimited.  By 
pinpointing  error- likely  situations  and 
designing  around  them,  almost  emy  error 
can  be  reduced  to  a  toleradale  level  .  .  .  . 
The  most  significamt  point  to  be  made 
eUsout  situational  changes  is  that  they  are 
relatively  permanent. 
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Optimal  Manned  Systems — 
The  Compatibility  Factor 


In  the  first  chapter  of  this  report  we  identified  cost, 
capability,  and  compatibility  as  the  three  impact  areas  that 
should  be  used  to  assess  the  value  of  human  factors  in  military 
system  development.  The  preceding  section  discussed  human  error  ' 
and  system  effectiveness  wherein  system  effectiveness  relates  to 
the  impact  areas  of  both  capability  and  cost.  This  section  will 
acquaint  the  reader  with  the  third  impact  area,  compatibility, 
via  a  discussion  of  optimal  manned  systems. 


.  .  .  where  those  systems  which  man  himself  plans, 
designs,  and  constructs  are  concerned,  I  submit  to 
you  that  there  is  no  such  thing  as  an  'unmanned 
system. '  It  must  be  appreciated  as  axiomatic  that 
all  such  systems  have  a  man  or  men  somewhere  in  the 
loop  between  planning,  attempting,  and  replanning. 

V  le.ther  the  question  is  one  of  foolproof  assembly, 
skillful  maintenance,  unerring  operation,  or  parrying 
counteraction,  man's  performance  in  relation  to  the 
equipment  which  is  invplved  will  decisively  affect 
the  accomplishment  of  the  'system.' 

The  corollary  to  this  axiom  is  that  the  impact  of 
man's  characteristics  must  be  taken  into  account 
in  the  design  of  the  equipment  if  the  system  is  to 
possess  maximum  probability  of  achieving  the  goal 
for  which  it  was  designed  in  the  first  place. 

(Flickinger  and  Hetherington,  1957) 

The  preceding  quotation  is  an  excerpt  from  a  paper  presented 
by  Brigadier  General  Don  D.  Flickinger  at  a  symposium  on  human 
factors  in  system  engineering.  The  point  is  just  as  valid  today 
as  it  was  when  made  over  20  years  ago.  Moreover,  there  is  such 
a  thing  as  an  "optimal  manned  system"  in  which  the  demands  of  the 
op®tational  system  exploit  man's  unique  capabilities  and  compen” 
sate  for  his  limitations,  in  other  words,  an  optimal  manned 
system  Is  one  in  which  man  is  most  compatible  with  his  designed 
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role,  functions,  tasks,  and  system  interfaces.  This 
"compatibility  factor"  and  the  notion  of  an  optimal  manned 
system  may  be  best  understood  by  considering  some  of  man's 
characteristics  in  a  system  setting.  Generalized  statements 
eUxjut  humam  capediilities  and  limitations  are  of  course  subject 
to  the  interpretation  of  specific  systems  and  environments. 
However,  the  generalities  offered  below  serve  to  establish  an 
awareness  of  the  compatibility  factor. 

Unique  Human  Cap2d)ilities 

In  complex  systems,  man  makes  the  most  significant  contri¬ 
bution  in  situations  where  all  of  the  performance  alternatives 
cannot  be  specified  in  advance  and  thus  pre-programmed.  Humans 
will  adapt  to  any  changes  in  the  system  input  and  environment. 
This  characteristic  is  mandatory  where  the  relations  between 
input  and  output  may  require  restructuring  in  the  course  of 
mission  accomplishment  and  where  all  operations  cannot  be 
reduced  to  logical >  preset  procedures. 

Man  makes  possible  a  more  diversified  system  mission. 

He  can  translate  uncertainty  into  probability  and  deal  with 
low  probability /high  value  exigencies;  and  he  can  develop  a 
"behavioral  strategy"  when  no  optimum  strategy  can  be  specified. 
His  ability  to  perform  a  variety  of  functions  and  to  utilize 
alternatives  means  more  is  accomplished,  including  multiple 
mission  performance,  recalleHsle  mission  attempts,  less  vulneredsle 
mission  accomplishment,  and  vehicles  returned  for  re— use. 

Man  has  the  ability  to  make  and  report  unique  observations 
,  including  observations  on  his  own  performance, 
observations  ph  system  performance,  observations  of  a  scientific 
nature,  and  incidental  intelligence. 
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Man  will  enhance  system  reliability.  Significant  human 
capabilities  which  may  not  be  easily  duplicated  by  a  machine  in 
so  small  and  reliable  a  package  include: 

'•  Selection  cunong  alternative  ways  of  achieving  a  mission; 

•  Integrating  a  large  eimount  of  information  gathered  from 
experience  and  bringing  it  to  bear  in  a  novel  situation; 

•  Sensitivity  to  a  wide  range  of  stimulus  patterns; 

•  Capability  to  detect  signals  through  noise; 

•  Capability  to  act  as  an  intermittent  servo  in  the  per¬ 
formance  of  a  number  of.  different  systems  or  equipment. 

Human  Limitations 

Man  comes  in  only  one  physical  model  and  can  only  be  inte¬ 
grated  into  the  system  concept  as  a  physical  whole,  with  certain 
general  characteristics  of  size,  weight,  shape,  strength,  etc. 

Man  has  certain  performance  limitations  such  as  sensitivity 
reaction  time,  nvunber  of  information  chemnels,  rate  of  operation 
environmental  stress  tolerance,  etc. 

There  is  a  definite  price  to  pay  for  mainteujiing  reliable 
performance  potential  in  man,  in  terms  of  training,  maintenance 
of  proficiency,  manuals  and  other  job  guides,  and  hvunem  factors 
design. 

Man  has  life  support  needs.  His  performance  deteriorates 
rapidly  when  these  physiological  needs,  such  as  nourishment, 
environmental  protection,  sleep,  comfort,  and  general  health 
maintenance  are  not  satisfied. 


/ 


Man  has  psychological  needs.  His  performance  usually 
deteriorates  over  prolonged  periods  of  high  stress  or  non¬ 
activity,  and  can  change  significantly  as  a  result  of  such 
psychological  variables  as  motivation,  frustration,  conflict, 
fecu:,  etc. 

Compatibility  and  User  Acceptance 

A  review  of  the  capeibilities  and  limitations  just  stated 
will  reveal  that  ccanpatibility  is  physical,  physiological, 
behavioral,  euid  attitudinal.  The  physical  and  physiological 
compatibility  is  basically  obvious  and  non-controversial. 

Man  cannot  be  expected  to  perform  if  he  cannot  fit  into  a  crew 
ccxnpartment  or  he  cannot  reach  the  controls  or  he  cannot  breathe. 
Behavioral  data  and  man's  sensory,  perceptual,  cognitive,  and 
motor  capabilities  have  to  some  extent  been  used  during  system 
development,  at  least  with  respect  to  human  engineering  of 
man-machine  interfaces.  However,  man's  attitudinal  system 
(i.e. ,  acceptance)  has  not  been  systematically  included  in 
man-machine  systems  design.  This  is  a  serious  error  as  a  highly 
motivated  man  can  compensate  to  s  considerable  extent  for  poorly 
designed  equipment  or  he  can  get  the  best  out  of  equipment  he 
likes.  It  is  true  that  if  a  system  is  designed  so  that  it  is 
easy  for  the  man  to  grasp  and  manipulate  the  controls,  and  if 
the  displays  are  easy  for  him  to  perceive  and  understand,  then, 
certainly,  the  system  will  be  more  acceptable  and  utilization 
will  be  enhanced.  Howevez:,  these  traditional  human  engineering 
efforts  are  not  sufficient  in  themselves  to  account  for  total 
acceptance.  A  man  dissatisfied  by  a  particular  system  design 
due  to  status,  economic,  or  survival  fears,  or  simply  a  desire 
to  operate  the  system  manually  because  he  enjoys  it,  may  not 
properly  use  equipment  which  has  been  designed  to  meet  all  other 
criteria.  He  will  reject,  underuse,  or  misuse  the  system, 
consciously  or  unconsciously.  Consequently,  system  effectiveness 
may  suffer  regardless  of  the  inherent  reliability  of  the  equipment 
pe?^  se. 


The  user  acceptance  issue  may  be  even  more  prominent  in 
advanced  systems  concepts  incorporating  extensive  automation. 

For  excunple,  one  approach  to  achieving  the  long-standing 
objective  of  highly  reliable  aircraft  landing  operation  under 
severely  degraded  visibility  conditions  lies  in  the  increased 
application  of  automatic  flight  control  techniques.  The  devel¬ 
opment  of  highly  reliable  automatic  control  systems  (such  as 
landing  systems)  by  the  best  engineering  talent  available  will 
not  solve  all  the  problems  associated  with  their  effective 
utilization  and  will,  in  fact,  create  new  problems.  For  many 
years  to  come,  such  complex  systems  will  be  man-machine  systems 
that,  at  a  minimum,  will  require  a  man  to  initiate  the  machine 
functions,  monitor  them,  and  decide  when  to  disengage  and  over¬ 
ride  them.  If  all  man-machine  interfaces,  including  the  user 
acceptance,  are  not  optimum,  system  effectiveness  cannot  be 
optimum. 

The  principal  attempt  to  optimize  interfaces  is  through  good 
human  engineering  design.  However,  traditional  human  engineering— 
usually  performed  after  the  system  has  been  designed  and  the 
breadboard  equipment  developed  by  engineers — has  been  applied  as 
if  man  were  perfectly  rational,  and  as  if  it  were  only  necessary 
to  consider  such  aspects  of  man  as  his  perceptual  2md  motor 
capabilities.  In  actual  fact,  however,  it  is  equally,  if  not 
more,  important  to  consider  man's  potential  attitudes  toward  the 
system  and  to  realize  that  these  attitudes  are  influenced  by  his 
fears,  anxieties,  aspirations,  euid  social  customs.  If  the  user 
acceptance  is  poor  and  performance  is  degraded,  then  coercion  or 
appeals  to  pride,  team  fellowship,  or  patriotism  may  serve  as 
poor  seconds.  On  the  other  hand,  if  systems  are  designed 
initially  with  high  acceptability  just  as  they  are  with  high 
reliability,  then  less  human  maintenance  is  required  just  as 
less  hardware  maintenance  is  required,  and  (in  optimal  manned 
system  is  possible. 


2-22 


In  suininary,  this  section  has  briefly  discussed  the  philo~ 
sophy  that  all  complex  military  systems  are  manned,  that  there 
is  such  a  thing  as  an  optimal  manned  system,  and  that  the 
compatibility  between  the  operational  system  demands  and  man's 
unique  capabilities  and  limitations— physical,  physiological, 
behavioral,  and  attitudinal — can  impact  system  performance  and 
cost.  For  an  optimal  manned  system  to  result  from  a  system 
development  program,  hunum  factors  considerations  must  be 
an  integral  part  of  the  acquisition  and  development  process 
throughout. 

The  first  part  of  this  chapter  has  attempted  to  provide 
a  simple  rationale  for  the  inclusion  of  human  factors  consider¬ 
ations  (by  human  factors  professionals)  in  system  development, 
the  remaining  sections  of  this  chapter  will  address  three  major 
implementation  questions: 

1.  Are  there  opportunities  for  human  factors  in  system 
' development? 

2.  Do  we  use  what  we  know  about  human  factors? 

Are  human  factors  contributions  affordable? 


3. 


Are  There  Opportunities  for  Human  Factors 
in  System  Development? 


It  has  already  been  offered  that  human  factors  as  a  pro¬ 
fession  has  data  and  methods  to  offer  which  will  effectively 
impact  system  cost,  capability,  and  compatibility. '  The  question 
remains  as  to  how  these  prescriptions  are  to  be  influential. 

One  might  even  question  the  feasibility  of  achieving  partial 
implementation.  The  technical  core  of  an  answer  to  these 
questions  is  presented  in  Chapter  3.  However,  it  is  a  useful 
transition  to  consider  the  opportunities  for  implementation  at 
this  point  frcan  a  preeminently  human  factors  point  of  view. 

Such  a  point  of  view  can  be  simply  summarized  by  a  table  taken 
from  Johnson  and  Baker  (1974).  It  follows  as  Exhibit  2-7. 
Basically,  this  chart  simplifies  the  weapon  system  acquisition 
process  and  identifies  a  number  of  human  factors  prcblems  (or 
requirements  for  human  factors  input)  during  the  development  of 
a  complex  system.  Chapter  3  provides  the  more  formal  and  more 
official  requirements. 


Given  that  there  are  opportunities,  are  human  factors  data 
eind  methods  used?  That  is  the  next  question.  ' 


Exhibit  2-7 

Human  Engineering  Problems  in  Weapon  Systems 


Stags  of  SystMit  Lift 

Souita  of  Problani  Raoognition 

PrafMration  of  Syttsm  Rsquirsmantt 

Raviaw  of  raquiiaments.  Abstract  and  analytic,  rathar 
than  ampiricaS.  Criteria  identification. 

Conduct  of  Datign  (or  Faasibtlity)  Study 

Study  of  altamativa  approaches. 

Preparation  of  outline  of  functions  to  ba  parfornwd. 
Oalinaation  of  data  relevant  to  these  functions. 

Preparation  of  flow  charts,  or  other  detailed  summariza¬ 
tion.  to  describe  the  functions. 

Performance  of  capabilities  analysis.  State-of-the-art/ 
Stata-of-tha-people  determ  ination. 

Davoiopmsnt  Planning 

Allocation  of  functions  within  the  definM  system 
boundaries  (man  can  do  better/computer  can  do 
better  determinations). 

Linking  together  the  functions  in  the  system.  Net  work 
determination. 

Assignment  of  functions  by  type  of  individual  involved. 

Oatign  of  Davalopmant  Modal 

Preparation  of  performance  descriptions;  task  analytic 
iob  dawription. 

Anaiysis  of  individual  workloads. 

Study  of  individual  interactions. 

Delineation  of  groups'  personal  mace  (by  furtction). 

Delineation  of  individuai’s  workspace  layout  within 
group. 

Determirtation  of  location  of  system  components. 

Study  of  alternative  personal  space  layouts. 

Analytis  of  human  information  requirenwnts. 

Anaiysis  of  human  responaa  requiramentiL 

' 

Design  of  system  interfaces. 

Determination  of  auxiliary  Job  supports. 

Defineation  of  procedures. 

Study  of  equipment  integration  for  simplification. 

Evaluation.of  Pretotypa  (Biaadboanfl  Modal 

Evaluation  throu^  mocfcups  and,  eventually,  prototype 

syittmia 

Production  Modal 

Evaluation  of  syslsm  (enginooring  and  proosdurot) 

JdMMM  Mnr.  tt74J 
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Do  We  Use  What  We  Know 
About  Human  Factors? 

The  true  "golden  era"  of  human  factors  occurred  from  the 
early  1950s  to  the  early  1960s  when,  among  other  events,  there 
were  always  four  or  five  major  electronic  systems  under  devel¬ 
opment  by  the  Air  Force.  From  the  mid-1960s  to  the  mid-1970s, 
however,  the  pattern  has  been  one  of  decline  and  deterioration. 

It  is  legitimate  to  ask  why. 

There  appear  to  be  many  reasons  why  humcin  factors  contri¬ 
butions  have  been  questioned  and  why  the  humeui  factors  knowledge 
has  not  been  applied  to  systems  or  accepted  by  system  sponsors. 
Scane  of  these  reasons  appear  to  be  organizational  factors, 
personnel  factors,  management  factors,  communication  factors, 
euid  a  host  of  other  factors  which  are  not  directly  germane  to 
this  project.  Nevertheless,  those  reasons  and  others  may  derive 
from  the  basic  problem  facing  human  factors  in  the  military: 
that  the  human  factors  researcher  and  practitioner  are  too 
frequently  called  in  after  system  design  and  development  has 
proceeded  to  a  point  where  costly  redesign  and  retrofit  is 
necessary  to  implement  human  factors . recommendations.  This  is 
the  biggest  complaint  of  researchers  and  practitioners  who 
believe  they  have  something  of  value  to  offer.  Therefore,  it 
seems  worthwhile  to  examine  in  detail  some  aspects  of  the  problem 
of  waiting  too  long  to  integrate  human  factors  into  the  weapon  ■ 
system  acquisition  process. 

First  of  all,  the  question  should  be  asked:  Is  this  really 
a  problem?  The  results  of  a  survey  (undated)  conducted  by  Meister 
and  received  in  July  1979  indicate  that  the  problem  still  exists. 
Meister  surveyed  the  three  major  participants  »rtio  determine  how 
much  human  factors  R«D  is  done  and  how  it  is  perceived.  These 
participants  are  RtD  laboratories,  R&O  contractors,  and  human 
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factors  practitioners  in  the  defense  industry.  Two  excerpts 
from  Meister's  paper  that  deal  with  the  practitioners'  answers, 
2md  one  from  his  Conclusions  section  serve,  we  believe,  to  make 
the  point  that  human  factors  is  largely  unaccepted  at  present 
in  system  design  and  development: 

.  .  .  Nor  do  design  engineers  tend  to  solicit  the 
assistemce  of  practitioners.  76%  of  respondents 
agreed  with  this  statement.  Again,  there  are 
individual  variations,  special  individuals  amd 
special  circumsteuices  but  the  armed  neutrality 
between  designer  and  practitioner  seems  the  same 
as  it  was  when  it  was  described  in  1967  (Meister 
emd  Farr) .  A  key  element  in  securing  designer 
cooperation  appears  from  respondents'  comments 
to  be  supportive  management.  A  number  of  factors 
appear  to  explain  the  designer-practitioner 
relationship:  the  designer's  wish  to  function 
with  complete  autonomy;  his  view  of  HP  require¬ 
ments  as  more  constraints  he  must  put  up  with; 
the  HF  group's  reputation.  It  is  helpful  if  the 
HF  group  has  sign-off  on  man-machine  interface 
drawings,  but  few  groups  have  this  sign-off. 

Slightly  more  than  half  (57%)  of  practitioners 
.  feel  that  there  is  still  considerable  resistance 
on  the  part  of  designers  to  the  inclusion  of  HF 
inputs  in  design.  The  positive  side  is  that 
almost  half  (41%)  do  not  agree  with  this  notion. 

It  may  be  that  these  responses  suggest  that  things 
are  improving  somewhat,  because  in  years  past 
almost  all  practitioners  would  have  given  negative 
answers  on  this  point .  Some  practitioners  feel 
that  jf  behavioral  inputs  are  reasonable,  engineers 
will  accept  them.  Unfortunately  some  HF  inputs  are 
inadequate  and  this  creates  resisteuice  to  or  rather 
avoidance  of  the  inputs.  Timing  is  all-important; 
inputs  made  after  decisions  have  been  reached  by 
designers  will  be  resisted. 

This  resistance  may  result  in  peurt  from  the  fact 
that  engineers  may  find  HF  inputs  to  design 
insufficiently  precise  and  quantitative.  72%  of 
the  practitioners  felt  this  to  be  the  case.  Some 
pointed  out  that  HF  data  must  be  translated  by 
practitioners  into  specific  design  terms  or  else 
the  input  is  merely  an  additional  burden  to  the 
engineer.  It  is  clear  that  there  is  continuihg 
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and  increasing  pressure  to  justify  the  utility  of 
behavioral  R&D.  While  this  may  not  be  unfortunate 
in  and  of  itself,  it  does  lead  to  a  number  of 
unfortunate  results;  faddism;  impatience  with  studies 
whose  effects  are  slow  to  emerge;  unwillingness  to 
invest  research  resources  where  results  are  risky. 


A  second  point  to  be  made  concerning  the  lack  of  human 
factors  integration  and  application  early  in  system  design  and 
development  can  be  drawn  from  a  consideration  of  the  military 
test  and  evaluation  program.  In  June  1978,  the  Office  of  the 
Under  Secretary  of  Defense  held  the  12th  Training  and  Personnel 
Technology  Conference  (TPTC)  with  the  topic  for  review  "Humem 
Factors  in  Weapon  Systems  Test  and  Evaluation  (T&E)."  The 
minutes  of  this  meeting  are  reported  in  Taylor  {1978).  Colonel 
Henry  L.  Taylor,  the  Executive  Secretary  of  the  Conference,  and 
Dr.  Jesse  Orlansky,  a  consultant  from  the  Institute  for  Defense 
Analysis  who  provided  written  comments  on  the  meeting,  make  some 
interesting  observations  concerning  tast  and  evaluation  and  human 
factors.  In  particular,  the  discussion  centered  on  information 
drawn  from  "The  FY  79  Department  of  Defense  Program  for  Research, 
Development,  and  Acquisition,"  1  February  1978,  Chapter  9,  Test 
and  Evaluation.  In  the  summery  of  the  minutes.  Colonel  Taylor 
pointed  out: 

1.  Sixty-one  major  progreuns  will  undergo  T&E 
in  FY  1979,  €md  DOD  will  monitor  a  total  of 
84  major  weapon  systems. 

2.  The  budget  request  for  T&E  in  FY  1979  is 
$3,683  million  for  development,  engineering 
and  testing,  and  $1,009  million  for  support, 
of  ranges,  test  facilities,  targets  and  joint 
tests,  for  a  total  of  $4.7  billion  for  T&E  in 

,  FY  79. 

3.  The  following  areas  are  now  being  emphasized 
by  the  DOD  T&E  prograun: 

a.  reduction  of  vulnerability  of  weapon 
systems 


b.  reliability  improvement  of  weapon  systems 

c.  greater  commonality  eind  standardization  of 
weapon  systems  among  military  services  and 
with  our  European  allies  (e.g.,  HARM,  STINGER, 
TRITAC,  JTIDS) 

d.  conduct  operationalv test  and  development  test 
earlier  in  development  cycle. 

Joint  Test  and  Evaluation  (JT&Es)  initiated  in  1972 
are  used  to  evaluate  the  effectiveness  of  a  weapon 
system  in  its  intended  operational  environment  and 
frequently  uses  the  forces  and  systems  from  two  or 
more  Services. 


Dr.  Orlansky,  referring  to  the  same  topic,  made  the  following 
"passing  comment" :  .  • 

.  .  .  the  magnitude  of  the  T&E  budget  ($4.7  billion 
in  FY  79)  does  not,  by  itself,  justify  a  larger  or 
smaller  Human  Factors  budget.  However,  $4.7  billion 
means  that  T&E  is  larger  (more  important?)  than  any 
category  of  RDT&E  (the  largest  is  6.4  Eng.  Development' 

$3.9  billion);  larger  than  any  Service  RDT&E  program 
(Air  Force  is  $4.3  billion);  larger  than  any  RDT&E 
authorization  title  (Naval  Vessels  is  $4.7  billion); 
etc.,  etc.  The  real  thought  is  whether  any  nominal 
Human  Factors  effort  could  produce  more  savings  than 
.  comparad>le  efforts  in  other  areas.  Perhaps  yes,. if 
someone  is  willing  to  explore  the  ^ssibilities. 


With  respect  to  the  comments  made  by  Colonel  Taylor  and 
Dr.  Orlansky,  above,  the  point  of  concern  is  whethet  or  not  early 
integration  of  human  factors  in  system  clevelopment  would  preclude 
some  of  the  problems  (and  enormous  costi )  associated  with  later 
test  and  evaluation. 


Two  final  observations  will  be  made  concerning  hxunan  factors 
which  are  "too  late  and  too  little,"  Filrst  of  all,  even  after 
systems  enter  the  operational  forces,  human  factors  problems  still 
exist  which  are  reported  as  deficiencies  and  result  in  costly 
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redesign  and  retrofit.  It  is  also  imperative  to  note  that  these 
deficiencies  are  not  just  gripes  about  sticky  knobs,  hard- to- read 
displays,  uncomfortable  crew  compartments,  etc.,  that  lead  tO' 
simple  degraded  performance  in  terms  of  time  and  error.  Rather, 
some  of  these  deficiencies  include  loss  of  life  and  costly  equip¬ 
ment.  As  an  indication  of  the  magnitude  of  these,  deficiencies. 
Exhibit  2-8  is  from  a  1978  briefing  given  by  the  Naval  Air 
Development  Center  on  its  human  factors  program.  While  these 
deficiencies  only  relate  to  three  weapon  systems,  the  absolute 
number  of  deficiencies  is  an  indication  of  the  magnitude  of  the 
problem. 

Exhibit  2-8 

Human  Factors  DeHciencies  Report 
for  Several  Navy  Aircraft  Weapon  Systems* 


F-14A  P38/C  8^ 


AIRCRAFT 


*1!"*  ?***  >*Jrcm  •  brM  pro»kM  by  Hw  Humw  Saeten  Cnginwrint  OlvMon  of  tbo  Naval  Air  Oavatapmant  Cantar. 


The  final  problem  observation  is  that  in  1977  DOD  announced 
a  tailoring  process  for  specifications  and  standards  in  Directive 
4120.21.  As  reported  by  Chaikin  (1978),  this  directive;  in  brief, 
permits;  (1)  selecting  documents  having  potential  application  to 
a  specific  procurement;  (2)  reviewing  those  potential  documents  to 
satisfy  only  those  clearly  applicable  to  a  contract;  (3)  imposing 
only  the  minimum  necessary  requirements;  and  (4)  examining  the 
surviving  requirements  to  tailor  or  adjust  the  provisions  so  that 
they  support  the  particular  system  involved.  As  Chaikin  further 
observed,  the  same  DOD  Directive  states  "beneficial  recommenda¬ 
tions  from  prospective  contractors  shall  be  solid bed  to  determine 
whether  additional  cost-effective  applications  and  tailoring  of 
cited  .  .  .  standard  .  .  .  requirements  can  be  accomplished  or 
cost-effective  substitutions  proposed." 

This  directive  provides  a  loophole  for  avoiding  human 
factors;  yet,  it  also  provides  a  basis  for  insuring  the  inclusion 
of  human  factors  if  they  can  be  established  to  be  cost  effective. 

In  summary,  there  appear  to  be  several  kinds  of  constraints 
on  the  successful  application  of  human  factors  to  military  systems 
development.  It  should  be  re-stressed  that  there  is  responsi¬ 
bility  on  both  sides — the  human  factors  side  and  the  progreim 
management  side — to  achieve  higher  levels  of  cooperation.  Most 
crucial  is, the  point  that  near-term  costs  (including  some  real 
conflicts  between  engineering  criteria  and  human  factors  criteria 
in  design  work)  can  result  in  long-term  gains  that  are  many 
multiples  of  the  near-term  cost. 
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Are  Hvunan  Factors  Contributions  Affordcdale? 

We  have  tried  to  indicate  by  means  of  the  preceding 
discussion  that  there  are  at  least  four  basic  eirguments  for 
the  inclusion  of  human  factors  in  military  system  development. 
Each  of  these  arguments  has  substantial  historical  roots  so 
that  they  are  a  part  of  a  standard  rationale.  In  brief,  these 
arguments  are: 

•  Life  cycle  costing  for  personnel  in  a  system  can  be 
impressive.  Personnel  must'  be  sustained  physiologically 
and  psychologically.  The  cost  of  addressing  personnel 
performance  through  training,  selection,  technical 
manuals,  and  other  performance  aids  is  expensive;  and 
the  cost  of  personnel  turnover  is  even  more  expensive. 

•  Complexity  remains  a  growing  problem  that  requires  a 
human  factors  contribution  for  its  amelioration. 

•  Design  deficiencies  cause  otherwise  avoidable  human 
error — such  errors  are  costly  not  only  in  terms  of 
lost  lives  and  lost  equipment  but  also  in  terms  of 
unfulfilled  missions. 

•  It  is  possible  to  conceive  and  implement  optimal 
manned  systems — systems  that  are  designed  to  utilize 
human  capedsilities  and  minimize  humem  limitations. 

The  response  to  these  argxanents  should  be: 

•  Human  factors  must  lead  to  an  increase  in  human 
reliedjility  and  consequent  increase  in  system 
■reliability. 
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•  Human  factors  must  be  applied  as  early  as  possible  in 
system  development  to  achieve  greatest  cost-benefit. 

The  cost  of  redesign  and  retrofit  of  weapon  systems 
where  human  factors  information  has  not  been  used  may 
far  outweigh  the  costs  for  human  factors  earlier  in 
system  design  and  development.  However,  this  must  be 
viewed  as  a  life  cycle  cost. 

•  Human  factors  value  will  derive  not  just  from  the 
accommodation  of  humcui  cap2d}ilities  and  limitations 
(which  would  typically  reduce  performance  time  cuid 
error),  but  also  from  better  equipment  utilization 
because  of  improved  attitudes.  If  user  acceptance 
is  not  considered  in  system  design,  the  system  may 

,  be  underused,  misused,  or  not  used  at  all— which 
could  result  in  mostly  costs  and  no  benefits. 

•  Effectively  integrated  htonan  factors  in  systems  will 
reduce  pther  costs  such  as  those  for  training,  selection, 
and  technical  manuals. 

•  Lack  of  human  factors  in  systems  will  result  in  damage 
to  the  equipkent  and  in  hazards  tb  the  user.  In  this 
casQi  incorporating  proper  human  factors  results  in 
cost  avoidance  through  avoidance  of  damage  to  equipment 
and  harm  to  personnel. 

•  Human  factors  in  syst^  design  and  development  will 
contribute  substantially  to  system  maintainability. 

In  general,  one  may  expect  (1)  time-to-maintain  to  go 
down,  (2)  maintenance- Induced  failures  to  go  down, 
a^d. (3)  spare  parts  consumption  to  go  down. 
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•  Human  factors  consideration  based  on  the  environment  in 
which  weapon  systems  are  to  be  operated  and  maintained 
will  insure  that  human  performance  is  not  seriously 
degraded  and,  thus,  that  system  performance  will  not  be 
seriously  degraded. 

•  Human  factors  integrated  into  the  entire  weapons  system 
acquisition  process  and  life  cycle  process  has  an 
intrinsic  value  because  it  becomes  part  of  the  hardware 
or  system.  Therefore,  the  effects  of  any  human  factors 
stay  with  the  equipment  or  military  service  amd  reduce 
the  cost  of  ownership.  This  is  in  contrast  to  training, 
in  which  the  investment  stays  with  the  individual  rather 
than  with  the  equipment. 

Conclusion 

It  is  important  for  policymakers  cuid  prograun  managers  to 
realize  that  good  human  factors  is  not  a  case  of  just  "proving 
the  obvious,"  i.e.  ,  that  hviman  factors  is  simply  common  sense. 

In  the  past,  a  common  sense  approach  has  produced  marginally 
acceptadjle  system  products  (from  a  human  factors  point  of 
reference)  based  on  the  fact  that  the  hardware  amd  technology 
associated  with  that  hardware  have  been  around  for  some  time. 
Experience  with  it  has  produced  a  level  of  knowledge,  one 
might  term  "lessons  learned" — which  is  really  the  common  sense 
to  which  we  refer.  In  periods  involving  quantum  leaps  in 
technology  and  hardware  sophistication,  as  we  have  been  experi¬ 
encing  for  some  time,  this  common  sense  breaks  down  due  foremost 
to  the  absence  of  "lessons  learned”  that  oomes  with  experience 
with  a  technology.  Human  factors  personnel'  have  training  and 
experience  to  bring  valuable  knowledge  emd  techniques  to  the 
system  development  process.  Human  factors  personnel  have  obtained 
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this  knowledge  due  largely  to  dealing  with  gaps  in  technology 
wherein  common  sense  has  broken  down.  In  addition,  operations, 
analysis  and  research  in  fields  such  as  system  engineering, 
aviation  medicine,  applied  physiology,  experimental  psychology, 
anthropometry,  and  sociology  have  contributed  a  great  deal  of 
basic  design  data,  which  human  factors  personnel  know  where  to 
find  and  how  to  interpret.  Perhaps  most  important  is  the  fact 
that  human  factors  personnel  have  the  necessary  motivation  to 
search  for  optimal  solutions  where  man  is  involved. 
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CHAPTER  3 


IDENTIFICATION  OF  THE  CONTRIBUTION  OF 
HUMAN  FACTORS  IN  SYSTEM  DEVELOPMENT 

In  order  to  identify  the  contributions  of  human  factors  in 
system  development  it  is  necessary  to  understand  both  the  major 
system  acquisition  and  development  process,  and  the  potential  for 
specific  human  factors  contribution  in  each  phase  of  this  develop¬ 
ment  process.  Therefore,  the  first  section  of  this  chapter 
summarizes:  the  major  system  acquisition  process,  requirement 
at  the  DOD  level  for  human  factors  R&D,  and  requirements  at  the 
service  level  for  human  factors  R&D.  The  material  in  this  section 
may  be  familiar  to  some  readers,  and  therefore  is  sufficient  as 
a  refresher.  However,  for  the  less  familiar  reader,  a  separate 
research  note  (ARI  RN-80-23)  is  available  which  provides  a  review 
of  essential  decision  points,  products,  directives,  and  other 
requirements  that  govern  system  development  2md.  enedile  humem 
factors  R&D.  This  docvsnent  may  be  obtained  through  DTIC. 

The. significant  conclusion  that  emerges  from  this  summary  emd 
review  is  that  there  is  an  adequate  and  formal  basis  existing 
for,  integrating  human  factors  R&D  into  military  systeiw,  but  in 
fact  this  is  not  being  dope. 

The  second  section  will  delineate  specific  human  factors 
efforts  in  each  system  development  phase  and  Indicate  how  these 
efforts  contribute  to  the  development '  of  the  principal  hximem 
factors  product  of  that  phase.  The  relationship  between  human 
factors  efforts/products  and  system  development  activities  is 
documented  . in  the  form  of  a  graphic  descriptive  model. 

The  remaining  five  sections  of  this  chapter  describe  in 
detail  the  human  factors  efforts  and  system  development  activities 
the  nature  and  content  of  the  principal  human  factors  product,  and 
an  example  of  h\iman  factors  contribution  for  each  development 
phase. 


Prior  to  addressing  these  major  topics  of  the  chapter  it 
is  necessary  to  reconfirm  the  notion  of  principal  human  factors 
products  in  systems  development.  In  Chapter  1  it  was  asserted 
that  there  is  indeed  a  principal  htiman  factors  product  that  will 
result  from  human  factors  efforts  during  each  major  phase  of 
system  development. 

Exhibit  3-1  delineates  these  products  for  each  phase, 
together  with  an  indication  of  their  potential  effect  on  system 
design.  These  products  were  essentially  derived  from  several 
studies  or  papers  concerned  with  concepts  or  models  for  suggesting 
what  human  factors  efforts,  decisions,  and  products  should  be 
undertaken  where  in  the  system  development  phase.  In  general, 
the  principal  human  factors  products  identified  in  this  report 
represent  a  reasonable  consensus  of  these  other  studies.  Some 
representative  docximents  from  which  these  products  were  developed 
include  Price  (1962);  Price,  Smith,  emd  Behan  (1964);  Erickson, 
Miles,  and  Secrist  (1978) ;  Goclowski,  King,  and  Ronco  (1978) ;  and 
Baker,  Johnson,  Malone,  and  Malone  (1979) .  The  rationale  and 
precedent  for  these  products  is  established  in  more  detail  in 
Appendix  A  for  the  interested  reader. 


Principal  Human  Factors  Products  for  Major  System  Development  Phases 
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Humein  Factors  R&D:  The  Existing  Basis 
in  System  Acquisition 


This  section  of  the  chapter  is  a  brief  overview  of  the  major 
system  acquisition/ development  process  and  the  existing  basis  for 
humcm  factors  R&D  at  the  DOD  level  and  at  the  service  level.  As 
was  mentioned  previously,  a  more  detailed  euialysis  of  this  topic 
is  provided  in  a  separate  research  note  available  through  the  DTIC 

The  Major  System  Acquisition  Process 

0MB  Circular  No.  A-109  (1976)  establishes  the  guidelines  and 
policies  for  major  governmental  acquisitions.  The  circular 
outlines  the  required  sequence  of  activities  through  which  the 
proposed  systeiv  must  pass,  and  specifies  the  key  decision  points 
at  which  the  evolving  system  must  gain  approval  before  the 
government  will  continue  to  fund  a  developing  system  or  to 
procure  any  new  major  system.  DOD  Directives  5000.1,  5000.2,  and 
5000.3  give  the  military  services  more  detailed  instructions 
in'  implementing  Circular  A-109  for  the  acquisition  of  major 
military  systems.  A  general  discussion  of  Circular  No.  A-109 
will  be  followed  by  a  short  discussion  of  the  directives. 

0MB  Circular  No.  A-109.  The  policies  in  Circular  No.  A*109 
attempt  to  systematically  integrate  the  various  factors  in  system 
development  and  to  avoid  past  problems  of  cost  overruns  emd 
premature  commitments  to  full-scale  development  and  production. 

To  accomplish  this,  the  circular  outlines  seven  activities  and 
specifies  four  major  decision  points.  Exhibit  3-2,  adapted  from 
OFPP  Peunphlet  No.  1  (1976)  ,  shows  these  activities  and  decision 
points.  The  boxes  describe  the  types  of  activities  involved  and 
the  numbered  circles  indicate  the  major  decision  points.  This 
acquisition  model  requires  an. identification  of  a  need  (Mission  ^ 
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Exhibit  3-2 

The  Activities  and  Decision  Points  of  OMB’s  Circular 
No.  A-109  Major  System  Acquisition  Cycle 


Analysis) ,  a  conparlson  between  the  present  technology  status  and 
what  is  needed  (Evaluation  and  Reconciliation  of  Needs),  a 
decision  to  continue  or  stop  @  ,  a  study  of  the  strengths  and 
weaknesses  of  alternative  systems  if  the  previous  decision  was  to 
continue  (Exploration  of  Alternative  Systems) ,  a  decision  to 
continue  or  stop  @  ,  a  demonstration  of  the  chosen  system (s) 
(Competitive  Demonstration) ,  a  decision  to  stop  or  continue  Q)  , 
the  building  and  test  of  the  complete  system  (Full-Scale  Develop¬ 
ment)  ,  a  decision  to  stop  or  continue  ,  the  production  of  the 
system  (Production) ,  and  field  use  of  the  system  (Deployment  and 
Operation) .  The  Department  of  Defense  directives  specify  the 
activities  within  the  phases  in  more  detail. 

Department  of  Defense  Directives.  The  Department  of  Defense 
(DOD)  Directives  5000.1,  5000.2,  and  5000.3  give  guidance  in 
implementing  0MB  Circular  No.  A-109  for  military  systems. 

DOD  Directive  5000.1  provides  policy  for  acquisition  of 
major  systems- -those  systems  exceeding  $75  million  for  research, 
development,  test  and  evaluation,  or  those  systems  exceeding 
$300  million  for  procurement.  Directive  5000.2  supplements  5000.1 
with  policies  and  procedures  for  the  DOD  system  acquisition 
process.  Directive  5000.3  gives  guidance  for  military  test 
and  evaluation.  These  three  directives  provide  for  five 
events  and  describe  the  activities  in  the  phases  between 
those  events.  Those  five  events  are  identification  of  mission 
needs.  Milestone  0,  Milestone  1,  Milestone  2,  and  Milestone  3. 

Because  the  circulars,  directives,  military  standards,  and 
service  regulations  used  various  nanws  for  the  same  phases,  it 
becama  necessary  to  adopt  standard  phase  and  decision  point  names 
to  avoid  confusion  when  using  information  from  the  different 
documents.  In  Exhibit  3-3,  the  major  system  acquisition  cycle 


model  of  Circular  A-109  is  again  shown.  Below  that  model  are 
the  standard  names  adopted  for  this  report.  Below  the  standard 
names  are  other  names  commonly  used  in  various  documents  to  refer 
to  the  same  phases. 

The  military  major  system  acquisition  model  as  described  by 
the  directives  is  in  Exhibit  3-4.  The  model  spans  the  time  from 
initial  threat  analysis  to  deployment  of  the  system. 

Revised  Department  of  Defense 
5000  Series  Directives 

Revised  Department  of  Defense  5000  series  directi\»es  were 
obtained  just  before  production  of  the  final  technical  report. 
These  latest  revisions  do  not  affect  the  case  that  can  be  made 
for  human  factors  R&D  requirements  in  the  military  acquisition 
process.  Rather  than  substantially  changing  the  relationship  of 
these  DOD  documents  to  requirements  for  human  factors  R&D  in 
systems  acquisition,  they  serve  instead  to  effectively  augment 
this  relationship.  Selected  excerpts  with  human  factors  R&D 
implications  are  shown  below  to  illustrate  the  characteristics 
of  the  new  directives. 

DODD  5000.1  Major  System  Acquisition  19  March  1980 
Objectives 

Integrate  support,  manpower,  and  related  concerns  into 
the  acquisition  process. 

Policy 

Affordability.  Affordability,  a  function  of  cost, 
priority,  and  availability  of  fiscal  and  manpower 
resources,  shall  be  established  and  reviewed  in  the 
context  of  the  PPBS  process  .... 


Exhibit  3-4 

The  Ptia:4;!>  uf  the  Military’s  Major  System  Aequisition  Model 


DEPLOYMENT 


DODD  5000.2  Major  System  Acquisition  Procedures 
19  March  1980 


Design  Considerations 

Manpower  and  Training 

(1)  New  systems  shall  be  designed  to  minimize  both 
the  numbers  and  the  skill  requirements  of 
people  needed  for  operation  and  support, 
consistent  with  system  availability  objectives 
Manpower  and  personnel  factors,  to  include 
n\imbers,  occupations,  and  skill  levels  of 
manpower  rer^uired,  shall  be  included  as 
considerations  and  constraints  in  system 
design.  Integration  of  manpower  and  personnel 
considerations  with  the  system  shall  start 
with  initial  concept  studies  and  shall  be 
refined  as  the  system  progresses  to  form  the 
basis  for  crew  station  design,  personnel 
selection  and  training,  training  devices  and 
simulator  design,  and  other  planning  related 
to  manpower  and  personnel. 

(2)  Where  applicable,  planning  for  training  shall 
consider  provisions  for  unit  conversion  to 
the  fielded  system  and  training  of  reserve 
component  personnel.  Such  planning  shall 
consider  tradeoffs  conducted  among  equipment 
design,  technical  publications ,■ formal 
training,  on-the-job  training,  unit  training, 
and  training  simulators  and  shall  develop  a 
cost-effective  plan  for  attaining  and  main¬ 
taining  the  personnel  proficiency  needed  to 
meet  mission  objectives. 

(3)  After  Milestone  0,  manpower  requirements 
shall  be  subjected  to  tradeoffs  with  system 
characteristics  and  support  concepts. 

Manpower  goals  and  thresholds  consistent 
with  projected  activity  levels,  maintenemce 
demands,  and  support  concepts  shall  be 
identified  by  Milestone  II.  Tradeoffs  for 
maintenance  effectiveness  eunong  manpower 
(nvimbers,  occupations,  and  skill  levels)  , 
support  equipment,  system  design,  and  the 
support  structure  shall' be  conducted.  The 
manpower  and  training  requirements  to 
support  peacetime  readiness  objectives  and 
wartime  employment  shall  be  developed  by 


Milestone  III.  These  requirements  shall 
be  based  upon  considerations  that  include 
available  Operational  Test  and  Evaluation 
results  and  current  field  experiences  with 
similar  equipment. 

Quality.  A  quality  program  shall  be  implemented 
in  accordance  with  the  criteria  and  procedures 
set  forth  in  DOD  Directive  4155.1  (reference  (j)) 
to  ensure  user  satisfaction,  mission  and  oper¬ 
ational  effectiveness,  and  conformance  to 
specified  requirements. 


DODD  5000.3  Test  and  Evaluation  26  December  1979 
Policies  and  Responsibilities 

Test  and  evaluation  (T&E)  shall  begin  as  early 
as  possible  and  be  conducted  throughout  the 
system  acquisition  process  to  assess  and  reduce 
acquisition  risks  and  to  estimate  the  operational 
effectiveness  and  operational  suitability  of  the 
system  being  developed.  Meaningful  critical 
issues,  test  objectives,,  and  evaluation  criteria 
related  to  the  satisfaction  of  mission  needs  shall  ' 
be  established  before  tests  begin. 

Before  the  Milestone  III  decision,  adequate  DT&E 
shall  be  accomplished  to  ensure  that  engineering 
is  reasonably  complete  (including  survivability/ 
vulnerability,  compatibility,  transportability, 
interoperability,  reliability,  maintainadjility , 
safety,  human  factors,  and  logistics  supportedsility) 
that  all  significant  design  problems  have  been 
identified,  and  that  solutions  to  these  problems 
are  in  hand. 

Attachment  1  -  page  2  Definitions 

Operational  Suitability.  The  degree  to  which  a 
system  can  be  satisfactorily  placed  in  field  use, 
with  consideration  being  given  availability, . 
compatibility,  transportability,  interoperability, 
reliability,  wartime  usage  rates,  maintainability, 
safety,  human  factors,  manpower  supportability , 
logistic  supportability,  and  training  requirements. 
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As  these  quotes  aptly  illustrate,  the  requirement  for  human 
factors  has  been  reinforced  with  more  direct  and  to-the-point 
statements  regarding  the  role  of  hviman  factors  in  system 
development. 

With  reference  to  DOD- level  requirements  for  human  factors 
R&D,  it  should  be  stated  that  DOD  maintains  one  human  factors 
specification  (MIL-N-46855)  and  one  human  factors  standard 
(MIL-STD-1472) ,  each  containing  extremely  detailed  human  factors 
requirementi 

Formal  and  Ii  ormal  Requirements  for 
Human  Factors  R&D  at  the  Service  Level 

Each  service  within  the  Department  of  Defense  maintains  its 
own  implementation  procedures  for  human  factors  requirements. 

These  requirements,  found  in  various  service  documents,  may  be 
broken  up  into  two  categories.  Human  factors  efforts  stated  as 
requirements  can  be  designated  formal  documents.  Formal  documents 
consist  of  the  service  regulations  and  instructions  (i.e..  Army 
Regulation  AR  602-1;  Air  Force  Regulation  APR  800-15;  eind 
Department  of  the  Navy  instruction  NAvmatinST  3900.9).  On  the 
other  hand,  human  factors  efforts  described  as  recommendations 
can  be  termed  informal  documents.  Informal  documents  consist  of. 
the  various  service  guidebooks,  handbooks,  manuals,  etc.,  that 
have  been  developed. 

The  following  points  can  be  made  about  these  servicie- level 
documents. 

•  The  trend  in  formal  documents  is  to  define  requirements 
and  responsibilities  for  human  factors  without  placing 
constraints  upon  the  methodology,  analysis,  and  data 
characteristics  used  in  research. 
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•  The  informal  documents  cover  primarily  those  topics 
not  promulgated  in  the  formal  requirements. 

•  The  consistent  thread  running  throughout  the  entire 
service  doctrine  is  the  application  of  humeui  factors 
as  a  total  system  concept,  encompassing  earlier  and 
more  detailed  involvement  in  major  military  system 
developments . 

Greater  detail  and  technical  explanations  of  these  formal  and 
informal  service- level  documents  may  be  found  in  the  aforementioned 
research  note.  The  following  sections  will  serve  to  illuminate 
specific  human  factors  R&D  efforts,  for  each  system  development 
phase,  which  are  considered  ideal  in  any  military  system 
development. 

Humem  Factors  R&D:  Specific  Efforts 
for  Each  System  Development  Phase 

There  is  an  ever-increasing  disparity  between  the  complexity 

I 

and  sophistication  of  modern  military  weapon  systems  and  the 
capabilities  of  the  military  personnel^  The  complexity  and 
sophistication  of  modern  military  hardware  is  increasing  at 
a  rate  greater  than  ever  before,  while  the  capability  of  our 
military  personnel  to  operate  and  maintain  these  systems  is, 
by  even  the  most  optimistic  accounts,  just  barely  keepinc  pace. 

This  hardware-personnel  mismatch  places  a  greater-than-e\'er 
burden  on  human  factors  to  contribute  a  favorable  impact  on 
capability,  cost,  and  compatibility. 

Humem  factors  provides  information  that  not  only  afiects 
equipment  design,  but  also  helps  determine  the  design  of  per¬ 
sonnel  selection,  training,  and  organizational  structures  that 
will  make  equipment  more  cost-effective  within  am  operational 
system.  Above  all,  human  factors  is  concerned  with  the  mission 
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objectives  of  an  equipment  system,  and  with  actions  that  will 
assure  the  cdsility  of  people  and  machines  to  meet  those  missioi^ 
objectives. 

Human  factors  should  begin  by  cmalyzing  mission  scenarios 
that  are  expected  to  be  encountered  in  combat.  Analysis  is 
performed  to  identify  the  critical  roles  men  will  play  to  succeed 
with  any  particular  mission.  Those  roles  will  be  broken  down 
into  functions,  which  can  then  be  simulated  or  performed  under 
the  expected  operational  conditions,  so  as  to  evaluate  equipment 
designs,  determine  manpower  requirements,  forecast  training 
requirements,  and  detect  the  organizational  structures  that  will 
be  required  to  support  the  equipment.  Alternatively,  prototype 
equipment  and  organizations  can  be  tested  empirically  by  field 
trials,  in  which  the  intended  user  population  attempts  to  perform 
the  required  mission  scenarios.  Such  tests  have  recently  been 
performed  for  several  pieces  of  equipment  during  the  operational 
tests  (OT-I,  -II,  or  -III)  required  by  the  Life  Cycle  System 
Management  Model.  They  have  provided  valuable  data  aU^out  equip¬ 
ment  and  organizational  design,  in  time  to  (1)  preclude  defects 
that  would  have  degraded  mission  perfozmemce,  and  (2)  prevent  the 
need  for  expensive  modifications  once  the  equipment  was  fielded. 

Finally,  human  factors  identifies  support  requirements. 

One  of  its  important  tasks  is  to  assure  that  such  needs  as 
maintenance  equipment,  training  programs,  and  training  devices 
are  ascertained  soon  enough,  so  that  when  the  equipment  is 
delivered  it  can  be  fielded  promptly  as  part  of  an  Integrated 
man-machine  system. 

Exhibit  3-5  shows  the  major  activities  of  the  system 
acquisition  cycle  on  a  time  line  from  Mission  Analysis  Phase 
to  Deployment.  Concurrent  with  the  system  acquisition  time 
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Exhibit  3-5 

Specific  Human  Factors  Efforts  and  Products  for  Each  Military  System  Development  Phase 


Exhibit  3-5  (Continued) 

Specific  Human  Factors  Efforts  and  Products  for  Each  Military  System  Development  Phase 
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Exhibit  3-5  ((Continued) 

Specific  Human  Factors  Efforts  and  Products  fur  Each  Military  System  Oevelopment  Phase 
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Exhibit  3-5  (Continued) 

Speciflc  Human  Factors  Efforts  and  Products  for  Each  Military  System  Development  Phase 
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Exhibit  3-5  (Continued) 

ific  Human  Factors  Efforto  and  Products  for  Each  Military  System  Development  Phase 


line  are  the  hum2m  factors  efforts  in  each  phase  and  their 
expected  outputs.  The  principal  human  factors  product  of  each 
phase  is  indicated  by  the  bold  type.  Together,  the  series  of 
charts  making  up  Exhibit  3-5  can  be  considered  a  descriptive 
model  of  human  factors  efforts /products  in  system  development. 


The  material  to  follow. discusses ,  for  each  phase  of  system 
development,  how  humam  factors  should  be  utilized  as  a  system 
progresses  through  each  phase.  The  humem  factors  efforts  within 
each  phase  will  be  described,  the  content  of  the  principal  human 
factors  product  of  each  phase  delineated,  and  examples  presented. 
The  objectives  of  human  factors  R&D  are  to  ensure  that  military 
equip^nt  and  organizations  are  well- fitted  to  hpman  users  and 
produce  the  maximum  advantage  in  terms  of  the  military’s  mission. 

This  model  is  a  generalized  one,  synthesized  from  many 
sources;*  it  is  non- service-specific,  and  applicable  to  a  wide 
variety  of  systems.  However,  the  model  cannot  and  does  not  , 
represent  a  rigid  process  of  development  for  ail  systems.  As 
with  any  model,  there  are  qualifiers.  While  the  order  of  human 
factors  considerations  should  be  maintained  for  .most  developing 
systems/  there  most  likely  would  be  variations  in  which  phase  the 
specific  human  factors  considerations  would  be  addressed. 

Depending  upon  the  complexity,  operational  environment,  crew 
role  and  size,  number  of  manufacturers,  etc.  involved  in  developing 
the  system,  the  resolution  of  specific  human  factors  efforts  may 
shift  from  one  phase  to  another.  In  other  cases,  the  developing 

*Principal  reference  sources  for  development  of  the  model, 
depicted  in  Exhibit  3-5  were:  Baker,  Johnson,  Malone^  6  Malone, 
1979;  Coburn,  1973;  Collins,  McGuiness,  Erl>  ::hman,  &  Bryce, 

1975;  Geddie,  1979;  Goclowski,  King,  Ronco,  &  Askren,  1978; 

Kaplan  6  Crooks,  1980;  Meister  6  Rabideau,  1965;  Merriman,  1976; 
MIL-H-46f*55B ;  Price,  Smith,  &  Behan,  1964;  Price  li  Tabachnick,  \ 
1968;  Van  Cott  6  Kinkade .  1972. 


process  may  be  compressed  or  specific  efforts  not  needed,  both  of 
which  could  result  in  entire  phases  being  eliminated.  In  reality, 
many  of  the  human  factors  efforts  are  iterative,  and  the  complex 
feedback  and  feedforward  loops  have  been  deleted  in  favor  of  a 
perceptually  and  conceptually  uncluttered  model. 

Mission  Analysis  Phase  Human  Factors 

When  a  threat  or  technological  breakthrough  has  been 
identified  and  a  decision  made  to  propose  a  new  system,  the 
various  system  developers  should  begin  a  coordinated  effort  to 
clearly  state  the  objectives  and  define  the  criteria  of  the 
system.  These  would  not  be  statements  on  how  to  accomplish  the 
mission,  but  rather  on  what  is  to  be  accomplished.  These  first 
activities  are  extremely  important,  as  the  criteria  will  become 
the  standards  for  r; equent  design  and  for  test  and  evaluation. 
Once  the  objectives  :.ave  been  identified  and  the  critc^ria  defined, 
thdse  other  political,  economic,  and  time  constraints  that  bound 
the  system  design  must  be  identified. 

The  human  factors  output  of  this  phase  is  the  determination 
of  the  role  that  m?n  will  play  in  the  new  system.  Will  man  be  an 
operator,  maintainer,  sensor,  manager,  analyzer,  decisionmaker, 
information  manager,  back'-up  to  equipment,  or  some  mix  of  the 
above?  A  very  important  decision  is  wheither  man  will  be  local 
or  remote  from  the  mission  equipment.  To  define  that  role,  all 
functions  that  are  needed  to  achieve  the  mission  objectives  must 
be  specified  first.  To  identify  all  functions,  the  operational 
and  environmental  conditions  under  which  the  system  iti  to  operate 
must  be  determined.  For  exeunple,  will  the  system  operate  in 
temperature  extremes ,  during  day  and  night,  in  unusually  rugged 
conditions,  or  for  unusually  sustained  periods  of  time,  etc.? 

In  addition,  analyses  of  existing  similar  systems  (if . any)  should 
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help  identify  operational  and  environmental  conditions  as  well  as 
other  positive  and  negative  aspects.  For  exeunple,  what  was  the 
role  of  man  in  the  predecessor  or  similar  system (s)?  What  mam 
functions  and  man-machine  functions  have  been  successful  and 
unsuccessful?  All  of  the  information  can  be  used  in  perforr^iing  a 
functional  analysis. 

Performing  tradeoff  studies  with  the  major  factors  , 

(e.g.,  logistics,  maintenance,  costs,  advamtages  anc  disadvamtages 
of  using  man  in  alternative  roles)  should  result  in  cost-effective 
system  configurations,  given  system  constraints.  Such  human 
factors  analyses  also  lower  the  probability  of  major  changes  in 
design  downstream  to  accomrooda-e  the  idiosyncracies  of  man. 

Human  Factors  Efforts  and  System 
Development  Activities  During 
Mission  Analysis 

This  subsection  provides  a  description  of  the  human  factors 
efforts  and  system  deveopi-ent  activities  shown  in  Exhibit  3-6. 

The  descriptions  are  keyed  to  the  numeric  code  on  the  chart. 

The  initiation  .^rocess  for  sv-^  st  ^'^quisition  is  based, 
first  and  foremost,  upon  the.' deriK’n'/^*. ,;?tion  of  a  need  for  a  system 
to  fulfill  a , to-be- specif ie^  r  »sion.  This  is  driven  from  one  of 
two  sources: 

(0.1)  Thrtat  Analyaia.  A  systematic  means  to  assess  enemy 
capability  in  relation  to  one's  own  capability  to  wage  war. 
Formally  defined,  it  is  stated  thus: 

The  process  employs  analytic  techniques  for  developing 

plausible  alternative  representations  of  foreign 

environnments  and  capabilities.  Thraatanalysis— 

1.  Provides  an  assessment  of  foreign  capabilities 
in  terms  of  combat  material,  employment  doctrine, 
environment  and  force  structure. 


2.  Provides  an  assessment  of  the  level  of  devel¬ 
opment  which  the  economy,  the  technology,  and 
the  military  forces  of  a  country  have  attained 
or  could  attain. 

3.  Includes  recasting  existing  intelligence 
assessments  and  forecasts  to  provide  statements 
of  the  threat  as  it  relates  to  a  specific  U.S. 
research  or  combat  development  project. 

(AR  3,81-11) 


(0.2)  Identifiaation  of  Technological  Breakthrough.  In  an 
attempt  to  tedce  advantage  of  the  possibilities  offered  by  newly 
or  nearly  available  technologies  an  effort  must  be  initiated. 

The  following  quote  tempers  the  application  of  new  technology. 

These  technical  and' scientific  advances  must  be 
evaluated  within  the  freunewor)c  of  the  military 
system  developments  to  insure  the  proposed 
development  is  relevant  to  the  DOD's  needs. 

(Adapted  from  DA  Pam  11-25) 


(O.Z)  Mission  Element  Needs  Statement  (MENS).  The  MENS  is 
a  DOD- level  required  product  of  the  Mission  Analysis  Phase  and  is 
aptly  summarized  by  the  following: 

The  MENS  pro’*ide8  justification  for  initiation  of 
further  system  acquisitioning.  progreunming,  addresses 
mission  related  def iOien.cies  (e.g.,  capabilities 
change  due  to:  change  in  enemy  threat  system 
obsolesences,  new  technology  availability) ,  pro¬ 
vides  guidance  for  system' concepts  and  identifies 
constraints. 

(Adapted  from  SECNAVINST  5000. lA) 


Requisite  human  factors  inputs  to  the  Mission  Analysis  Phase 
should  be  found  in  the  MENS,’  which  is  the  subject  of  the  preceding 
discussion.  The  discussion  to  follow  is  the  substance  of  human 
factors  inputs,  and  culminates  in  the  major  human  factors  product 
in  the  Mission  Analysis  Phase — the  role  of  man  in  systems. 
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(0.31)  Define  System  Objectives.  Human  factors  inputs  must 
be  general  in  nature  by  definition.  Thus  a  general  but  valid 
statement  of  the  mission(s)  and  threat  must  be  developed 
(e.g.  ,  potential  targets,  enemy  weapon  capcibilities)  . 

(Adapted  from  MIL-STD-490) 

The  specification  of  mission  requirements, 
objectives  auid  criteria  include  the  determination 
of  the  class  system  cind  hardware  involved  cuid  a 
statement  of  the  activities  assigned  to  a  system 
envisioned  for  a  specific  missionCs). 

(Adapted  from  Kaplan  &  Crooks,  1980) 

Once  having  defined  system  criteria,  additional  descriptions 
must  be  prepared  that  provide  a  refinement  of  mission  details. 
These  include  determination  of  mission  constraints  and  limitations 
that  impact  the  feasibility  of  continued  system  development. 

(0.32)  Determine  Operational  Conditions.  This  includes  the 
specification  of  two  items:  (1)  system  operation  and  (2)  condi¬ 
tions  of  performance.  Both  iire  important  to  mission  feasibility. 

System  operating  characteristics  and  goals  must  be  identi¬ 
fied  to  determine  a  probability  for  mission  success.  Personnel., 
crew,  and  hardware  characteristics  during  performance  of  a 
mission  must  be.  defined  to  illustrate  mission  constraints  that 
impact  mission  success. 

(0.33)  Determine  Environmental  Conditions,  Known  system 
characteristics,  operating  locus,  and  enemy  countermeasures  must 
be  identified  to  reveal  limitations  upon  various  approaches  to 
a  mission.  This  includes  a  determination  of  terrain  and  climate/ 
weather  conditions. 


(0.34)  Analyze  Similar  Systems.  Since  most  new  systejms 
rarely  make  use  of  technology  never  before  experienced  in  any 
total  sense,  the  analysis  should  encompass  previously  encountered 
strengths  and  weaknesses  as  well  as  lessons  learned  to  avoid 
previous  development  pitfalls. 

A  process  of  synthesis  must  be  utilized  to  incorporate  the 
findings  of  the  various  analyses  previously  conducted  in  order  to 
determine  the  overall  feasibility  of  a  system  development  program. 
In  addition  to  system  characteristics,  this  process  should  lead 
ultimately  to  a  defined  role  for  man — the  master  product  of  human 
factors  in  mission  analysis  (and  the  sole  human  factors  reason 
for  participation  of  specialists  at  this  stage) . 

(0.35)  Conduct  Function  Analysis.  Function  analysis, 
originally  conceived  as  a  process  in  system  engineering  to  select 
functional  categories , for  system  performance  (involving  the  tech¬ 
niques  of  functional  flow  diagraunming  and  block  diagramming) ,  has 
been  extended  to  include  the  functions  of  man  in  the  system, 
without  whom  no  system  performance  would  be  possible.  This 
analysis  involves  the  seJ.ection  of  manual,  hardware,  or  automated 
performance  for  each  fun<ition.  The  analysis  stops  short  of 
allocating  specific  funct;ions  %o  mem  or  machine,  but  terminates 
with  data  tantcunount  to  euch  a  distinction.  For  human  factors, 
the  interest  was  exclusi\'ely  in  assessing  the  capediilities  of 
humans  in  emy  specific  system. 

(Portions  adapted  from  MlL-STD-490) 

(0.36)  Role  of  Man  Defined.  The  culmination  of  the  pre¬ 
ceding  activities  is  the  role  of  man  in  systems  (from  a  human 
factors  standpoint).  Since  this  is  a  major  product  of  mission 
analysis,  its  principal  components  have  already  been  indicated 
and  will  not  be  repeated  again. 
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Content  of  the  Role  of  Man  Statement 

A  statement  of  the  role  of  man  as  part  of  the  Mission 
Element  Needs  Statement  (MENS)  should  include  the  following 
considerations: 

Assvunptions: 

•  A  separate  "role  of  mem"  analysis  will  be  provided  for 
each  alternative  system  concept  selected. 

•  Human  engineers  will  develop  "role  of  man"  concepts  and 
interact  with  mission  analysis  tecsm  in  development  of 
MENS. 

•  "Role  of  man"  components  are  listed  according  to  probable 
order  of  presentation  in  MENS  (not  according  to  their 
development  sequence) . 

Actions : 

1.  List  effects  envisioned  for  overall  system  as  a  result  of 
role  of  man  devised  for  each  alternative  system  concept 
as  configured  (e.g.,  operability,  maintainability,  mission 
effectiveness) . 

_ 2,„List  effects  envisioned  for  man's- role/personnel 

subsystem  as  a  consequence  of  each  alternative  system 
concept  as  proposed  (e.g.,  safety,  habitability,  user 
acceptance) . 

3.  Detemnine  location  of  man  in  system  to  perform 
designated  role. 

4.  Specify  advantages  accorded  m^m's  role  for  each  alter¬ 
native  concept  (e.g.,  facilitate  operation  of  system, 
allowance  for  contingencies) .  ' 
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5.  Specify  disadvantages  accorded  man's  role  for  each 
alternative  concept  (e.g. ,  manpower  reserves  consumption, 
level  of  training  requirements). 

6.  Determine  required  human  performance,  behaviors, 
capabilities,  and  performance  limits  (e.g.,  sensing, 
processing,  information  storage,  decision  making, 
responding)  identified  for  each  functional  category. 

7.  Determine  personnel  constraints  impacting  main's  role  for 
each  alternative  system  concept  such  as  the  following: 

maocimum  and  minimum  numbers  of  personnel  who  can  be 
used  in  the  system 

types  of  personnel  (e.g.,  skill  level  and  aptitude) 
available  for  system  assignment 

anthropometry  of  identified  personnel  population 
(existing  amd  pfoji:cted) 

user  acceptance  problems  projected  and  their  effects 

effects  of  system  and  mission  as  configured  on 
personnel  vulnerability  (e.g.,  environmentdl  hazards) 

communication  requirements  and  limits  (system  and 
other  personnel). 

8.  .Je»-ermine  implications  envisioned  for  each  alternative 
system  concept  upon  requirements  for: 

a.  training  (e.g.,  level  of  training,  trainability , 
training  support  and  facilities,  training  devices) 


a. 

c. 

d. 

e. 

f. 


b. 


manpower  (e.g. ,  manpower  levels,  performance 
availability) 

c.  life  support 

d.  "-ilities"  support  (e.g.,  logistics,  reliability, 
maintainability) 

e.  social/orgamizational  impact  (e.g.,  MX  basing). 

9.  Select  contributions  to  function  analysis  in  Mission 
Analysis  Phase: 

V 

a.  identification  of  threat 

b.  need  demonstration:  new  system  or  modification  to 
current  system 

c.  requirement 

d.  mission 

e.  system  objective  definition  (and  required  input/ 
output) 

f.  mission  segment 

g.  scenario (s) 

h.  functional  categories 

i.  functional  flow  and  operational  event  sequences 

j.  system  specification: 

1.  meuiual 

2.  hardwired 

3.  automated:  Facilitate  system  functioning 

Override  (bypass)  system  malfunctioning 
Control  system,  graceful  degradation 
Permit  system,  to  operate. 
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10.  List  humeui  factors  characteristics  that  will  facilitate 
successful  system  development  and  mission  success  for 
each  alternative  concept  (design,  development,  testing, 
production,  deployment,  and  operation) : 

a.  advancement  in  state-cf-the-art  human  factors 
technology 

b.  currently  availcible  human  factors  technology. 

,  11.  List  impacts  upon  cost  and  system  effectiveness  for  each 
alternative  concept  in  association  with  human  factors 
inputs : 

a.  R&D,  training,  personnel,  manpower 

b.  mission  success,  vulnerability,  survivability. 

12.  Prepare  Human  Factors  R&D  Program  Plan  tailored  to  each 
alternative  concept  for  balance  of  system  life  cycle. 

Mission  Analysis  Phase;  Example 
of  Human  Factors  Contributions 

Both  doctrine  and  recent  history  suggest  that  future 
warfare  will  require  a  capability  to  fight  at  night,  and  during 

operations  sustained  around  the  clock.  The  Soviets  claim  an  _ 

ability  to  fight  at  night.  Aside  from  that  threat,  the  importance 
of  night  combat  is  obvious.  Modern  battle  will  move  rapidly; 
the  unit  that  move  and  fight  at  night  may  gain  a  permanent , 
advantage.  Furthermore,  weapons  are  so  lethal  that  maneuvering 
at  night  may  have  great  advantages  in  security.  But  unfortunately 
little  is  known  about  the  capabilities  amd  limitations  of  soldiers 
during  the  night  or  in  continuous  operation.  We  lack  knowledge 
of  the  mission  scenarios  that  would  be  involved  £urid  therefore  of 
how  to  train  for  battle  at  night.  There  is  a  requirement  for 
research  on  which  to  base  appropriate  training;  the  development 
of  tactical  doctrine,  and  plans  .for  unit:  rotation,  performance 
aids,  and  training  devices. 
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An  example  is  provided  by  the  work  of  the  9th  Infantry 
Division  at  Fort  Lewis,  Washington,  which  was  concerned  with 
improving  the  ability  of  individual  soldiers  to  navigate  at 
night. 

Army  researchers  began  by  analyzing  the  strategic  goals 
of  night  operations  and  continuing  operations.  From  these  goals 
they  identified  specific  tactical  missions,  analyzed  the  situa¬ 
tions  that  would  occur,  and  then  determined  the  human  behavioral 
requirements  for  mission  success.  This  produced  a  statement  of 
the  behavioral  requirements  for  mechanized  infantry  squads  and 
platoons  in  a  combined  arms  defense  against  a  deliberate  break¬ 
through  attack.  A  methodology  was  developed  in  which  a  niamber  of 
specific  mission  scenarios  can  be  varied,  so  that  the  soldier's 
tasks  occur  under  a  variety  of  conditions  (differing  levels  of 
light,  fatigue,  stress,  out-of-phase  daily  rhythm).  Tasks 
critical  to  the  enemy's  defeat  were  evaluated  ir  terms  of  what 
behaviors  they  required  compared  with  what  soldiers  were  actually 
aole  to  perform.  Areas  in  which  soldiers  fell  short  were  identi¬ 
fied  and  recommendations  were  made  concerning  changes  to  tactics, 
equipment,  and  training. 

Army  researchers  studied  several  basic  issues,  including  the 
ability  of  soldiers  who  are  moving  in  vehicles  to  maintain  their 
orientation.  It  was  found  that  even  infantrymen  on  foot  had  a 
poor  sense  of  direction  at  night,  and  lost  their  navigational 
reference.  Those  in  personnel  carriers,  and  the  crews  of  armored 
vehicles,  were  substantially  poorer  in  this  regard.  This  fact  is 
of  special  interest  because  current  vehicles  do  not  carry  naviga¬ 
tional  aids.  Study  of  individual  differences  found  no  way  to 
select  people  based  on  their  backgrounds,  but  suggested  that 
actual  experience  in  land  navigation  did  improve  both  night  and 
day  navigation.  Further  research  is  how  in  progress. 
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Meanwhile,  light- attenuating  devices  CLADs)  were  identified 
as  a  possible  tool  for  training  in  nighttime  operations. 

Ordinary  darkened  lenses  posed  a  hazard:  They  attenuate  light 
well  only  in  the  visible  range,  and  transmit  harmful  infrared  and 
ultraviolet  light.  Since  the  pupil  of  the  eye  responds  only  to 
visible  light,  the  eyes  of  the  subjects  wearing  ordinary  goggles 
would  be  dilated,  maiking  tiiem  particularly  vulnerable  to  damage. 
By  using  lenses  with  an  appropriate  degree  of  attenuation,  it  is 
possible  to  closely  approximate  visual  conditions  under  different 
degrees  of  moonlight  or  darkness.  The  Army  evaluated  the  goggles 
at  Fort  Rucker  in  flying  training  and  at  Fort  Lewis  in  land 
navigation.  Test  results  suggested  a  satisfactory  behavioral 
approximation  of  darkness,  and  research  continues  concerning 
their  suitability  for  teaching  various  nighttime  tasks. 

The  use  of  LADs  has  been  demonstrated  fof  the  training  of 
tank  drivers  at  Fort  Knox,  for  infantry  navigation  at  Fort  Lewis, 
and  for  nap- of- the- earth  night  helicopter  flying  at  Fort  Rucker. 
Training  of  this  kind,  conducted  by  day,  has  great  advantages  in 
safety.  More  training  can  be  accomplished  for  the  time  invested 
because  supervisors  and  trainers  can  direct  the  activity  using 
normal  vision,  and  because  logistics  is  not  impeded  by  darkness. 

The  Basic  Combat  Training  Group  at  Fort  Jackson  has  imple¬ 
mented  night  rifle  training  using  the  LAD.  Aside  from  the 
general  difficulty  in  seeing,  soldiers  tend  to  overestimate  range 
at  night — cin  error  which  training  can  correct. 

This  research  has  provided  a  limited  cap^d}ility  to  train 
infantrymen,  truck  drivers,  cmd  helicopter  pilots  in  night  oper¬ 
ations.  It  provides  an  initial  scientific  understemding  of 
soldier  performance  at  night,  and  contributes  to  building  a  human 
technology  data  base  to  help  develop  methods  and  machinery  to 
counter  a  night  fight  threat  and  maximize  our  night  fight 
capabilites. 
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Concept  Development  Phase  Human  Factors 

In  the  Mission  Analysis  Phase,  the  system  functions  were 
identified.  In  the  Concept  Development  Phase,  criteria  for  each 
function  are  derived  and  rank-ordered  or  weighted  in  importance. 
For  example,  in  a  particular  function  one  must  determine  the 
relative  importance  of  accuracy,  flexibility,  and  firing  rate. 

In  one  system  a  fast  firing  rate  is  roost  important;  wiiile  in 
another  accuracy  may  be  ranked  first. 

After  rank-ordering  or  weighing  the  criteria,  the  criteria 
are  compared  to  human  capabilities.  The  comparison  process 
should  suggest  which  functions  should  be  allocated  entirely  to 
machines,  others  entirely  to  man,  and  still  others  to  some  man- 
machine  combination.  Those  man-machine  functions  should  be 
further  studied  to  determine  which  combinations  would  prodr.ce 
the  most  effective  performance.  It  is  equally  poor  to  have  a 
man  do  a  job  at  which  a  machine  is  better  as  it  is  to  have  a 
machine  do  a  job  at  which  a  man  is  better.  The  intent  of  the 
man,  machine,  and  man-machine  function,  allocation  is  to  have  the 
most  effective  participation  of  man,  given  the  system  constraints 

Human  Factors  Efforts  and  System 
Development  Activities  During 
Concept  Development 

This  subsection  provides  a  description  of  the  hxaman  factors 
efforts  and  system  development  activities  shown  in  Exhibit  3-7. 
The  descriptions  are  keyed  to  the  numeric  code  on  the  chart. 

The  major  products  of  the  Mission  Analysis  Phase,  which 
includes  the  Role  of  Man  in  Systems,  are  ceurried  forward  as  input 
to  the  Concept  Development  Phase,  wherein  the  initial  allocation 
of  functions  between  men  and  machines  occurs.  Steps  in  system 
development  include  the  following  in  concept  development. 


Elxhibit  3-7 

Spefifii-  Human  Factors  Efforts  During  the  Concept  Development  Pfias*'  (1.) 
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(1.1)  Assign  Program  Manager.  Inherent  With  the  desig¬ 
nation  of  a  system  developm-int  progr^un  manager  is  a  cluster  of 
activities  involved  wit^  Tnana'''em2nL  of  the  development  life  cycle 
(e.g..  Development  Plan,  Decision  Coordinating  Paper,  etc.). 

(1.2)  Study  Alternatives.'  This  general  title  includes 
the  initiation  of  development  of,  and  culminates  with  the  veri¬ 
fication  of,  the  conceptual  system(s) .  It  involves  the  continued 
refinement  of  Mission  Analysis  Phase  products,  validation  of  the 
same,  and  development,  study,  and  approval  of  alternative  system 
concepts  which  are  summarized  in  a  Decision  Coordinating  Paper 
(DCP). 

The  principal  human  factors  activities  in  the  Concept 
Development  Phase  are  directed  toward  the  allocation  of  functions 
to  man  and  machine.  The  activities  involved  in  arriving  at  this 
product  are  the  subject  of.  the  following  discussion. 

(1.21)  Apply  Criteria  in  Their  Order  of  Importance .for 
Each  Function.  This  is  a  two-part  activity.  It  involves 
(1)  development  of  appropriate  tradeoff  criteria  and  (2)  appli¬ 
cation  of  the  tradeoff  criteria  to  the  allocation  of  functions 
betweeh  men  and  machines. 

Examples  of  candidate  criteria  include! 

•  cost  (procurement . and  operation)  . 

•  weight 

•  development  time 

•  daveloptaent  ris)( 

•  safety 

•  maintainability 

•  system  effectiveness  prediction 

•  physical  volume,  size  limits 

e  survivability. 
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And  for  hunan  factors  specifically: 

•  human  performance  capabilities  and  limitations 

•  machine  performance  capabilities  and  lim.itations 

•  special  case  effects  of  automation  upon  system  capability. 

Application  of  criteria  can  involve  manual  or  computer-aided 
models,  but  must  demonstrate: 

•  effects  of  the  system  upon  human  performance 

•  effects  of  human  performance  upon  system  effectiveness 

•  rationale  of  the  decisions  based  upon  criticality  of 
alternative  versions. 

(Adapted  from  Coburn,  1973) 

(1.22)  Compara  Human  Capabilities  to  Function  Criteria. 

After  obtaining  a  preliminary  allocation  of  functions  to  men 
and  machines,  an  assessment  must  be  made  of  human  capability  to 
perform  effectively  each  function  designated  to  man.  This  may 
involve  lessons  le^u:ned  and  other  data  obtainable  through  analysis 
of  previous  similar  systems  or  through  human  performance  reliability 
simulation  by  means  of  a  number  of  currently  available  models. 
Whatever  method  is  utilized,  the  result  must  be  a  reuik  order  of 
preferred  candidate  manned  functions  along  with  a  rationale  for 
their  choice. 

(1.2S)  Explore  Possible  Man-Machine  Combinations  to  Achieve' 
Function  Endpoint.  The  final  activity  in  the  analytic  process  of 
function  allocation  requires  the  analysis  of  candidate  allocation 
versions  (or  man-machine  combinations) ,  utilizing  models  such  as 
those  described  above,  but  with  the  intent  of  assessing  more 
comprehensive  performances  (such  as  wor)<:load  characteristics) 
rather  than  individual  functions.  The  method  utilized  should 
provide  a  decision  and  associated  rationale  for  humem  factors 
choice  of  man-machine  combinations. 
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(1..24)  Function  Allocation.  The  principal  product  of  the 
Concept  Development  Phase  is  the  allocation  of  function  to  man 
and  machine.  This  product  is  described  in  great  detail  elsewhere 
and  will  not  be  repeated  here. 

(1.3)  Decision  Coordinating  Paper  (DCF).  DCPs  are  docu¬ 
ments  that  support,  authorize,  and  promulgate  decisions  to 
initiate  development  progr2uns  and  establish  appropriate  Advanced/ 
Engineering  Development  Line  items  (OPNAVINST  500Q.42A) .  They 
present  the  rationale  for  starting,  continuing,  reorienting,  or 
stopping  a  major  development  program.  DCPs  address  affordability 
of  a  proposed  system  as  well  as  other  important  factors 

(e.g.,  threat,  risks,  acquisition  cost,  strategy,  and  performance 
paraunecers  for  evaluation)  . 

(Adapted  from  DA  Pam  11-25) 

(1.4)  DSARC  I/(S)SARC  I.  Defense  System  Acquisition 
Review  Council  1/ (Service)  System  Acquisition  Review  Council  1 
provide  recommendations  as  to  the  status  and  readiness  of  each 
major  system  under  development  to  advance  to  subsequent  phases 

in  its  life  cycle.  They  review  such  documents  as  the  DCP  in  this 
process.  Final  decisions  are  made  by  the  Secretary  of  Defense  or 
his  designee. 

Content  of  the  Allocation  of  Functions 
to  Man  Statefient  as  Part  of  the 
Decision  Cooi'dinatinq  Paper 

A  staterient  of  the  allocation  of  functions  to  man  as  part 
of  the  DCP  stiould  include  the  following  considerations: 

Assumptions : 

e  The  following  items  will  provide  direct  input  to  the 
specification  of  the  function  allocation  piocess: 
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-  Mission  Element  Needs  Statement  {MENS) 

-  mission  scenarios 

-  functional  flow  block  diagrams 

-  mission  time  lines. 

•  Function  allocation  will  provide  support  to  the  proposed 
system  by  illuminating  the  following  criteria: 

-  system  performance 

-  cost-effectiveness. 

Both  criteria  have  as  a  function  human  performance. 

Human  performance  can  be  specified  according  to  degree 
of  detail  available  about  the  system  mission  and  envi¬ 
ronmental  factors. 

•  Function  allocation  will  detail  functions  involving  both 
operators  and  maintainers. 

•  The  following  general  process  is  assumed  for  the  function 
allocation  process: 

-  identify  and  allocate  tasks  and  functions  to  be 
assigned  to  all  personnel 

-  identify  required  equipment 

.  -  evaluate  selected  man-machine  combinations 

-  arrange  tasks  and  .  functions  to  maximize  mission 
effectiveness  and  reliability. 

Actions  I 

e  This  section  is  arranged  according  to  a  topical  develop¬ 
ment  sequence  for  function  allocation  (not  development 
sequence) . 
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1.  Specify  human  factors  criteria  selected  for  allocation 
of  functions  (e.g.,  response  time,  error  rate  or  human 
performance  reliability,  cost). 

2.  Specify  other  criteria  selected  for  allocation  of 
functions  (e.g.,  cost,  personnel  cost,  required  training 
weight,  development  time,  development  risk,  safety, 
maintainability,  system  effectiveness,  physical  volume 
and  size  limits,  and  survivability). 

3.  List  allocation  of  each  function  to: 

a.  one  or  more  opera tor s/maintainers 

b.  machine  only  (includes  automation) 

c.  combination  of  man  and  machine 

d.  function  currently  not  eunenable  to  man  or  machine 
performance. 

4.  Multiple  operator/maintainer  and  man/machine  functions 
will  include  specif icaition  of  the  type  of  redundancy  in 
the  task  being  proposed  (e.g. ,  parallel  or  sequential 
mode,  or  hybrid  of  both). 

5.  Provide  estimate  of  feasibility  of  performance  for  each 
function  allocated ^  .  List  the  effect  of  different  allo¬ 
cation  versions  upon  mission  success  (e.g.,  probability) 
Provide  estimate  of  workload  upon  operators/maintainers 
as  a  result  of  each  allocation  version  (at  least, 
nominally).  (At  this  level  of  development,  workload 
implies  task  difficulty  and  will  include  requirements 
for:  precision,  concentration,  criticality,  mission 
priority,  and  task  continuity  for  operators/maintainers 
involved  in  each  manned  function.)  Account  for  effects 
of  user  acceptance  for  each  allocation  version. 
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6.  List  human  performance  capabilities  required  of 
operators/maintainers  for  each  function  involving  man 
and  verify  whether  or  not  man  can  perform  each  in  terms 
of  required  physical  and  mental  parameters  over  the 
required  time  period  and  within  tbs  anticipated 
environment. 

7.  Prepare  rank  orders  for  candidate  allocation  combina¬ 
tions  according  to  criticality  of  functions.  (Criteria 
for  criticality  will  also  be  specified.) 

8.  List  all  bottlenecks,  data  overloads,  acceptance 
problems,  and  other  mission-critical  faults  that  occur 
as  a  consequence  of  each  allocation  version.  Specify 
the  means  by  which  each  allocation  version  will  relieve 
them  and/or  how  to  modify  the  allocation  version  to 
accommodate  them. 


9.  Prepare  a  comparison  matrix  which  exhibits  all  alloca¬ 
tion  versions  versus  the  selection  criteria  (entries  in 
the  matrix  are  estimates  of  absolute  performance  or  rank 
for  each  allocation  version  or  each  criterion  measure) . 

10.  List  preferred  manned  funbtions  as  well  as  other 
combinations  or  allocated,  versions . 

11.  Provide  a  rationale  for  thp  preferred  approach  and 
selection  to  justify  the  allocation. 


Navy  fliers  are  faced  with  too  many  tasks  requiring 
the  use  of  eyes  and  hands.  Excessive  workload  can  have 
serious  adverse  effects  on  mission  effectiveness  and  safety. 
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The  workload  must  be  reduced  if  the  performance  potential  being 
designed  into  new  systems  (and  which  the  operator  is  now  too  busy 
to  fully  utilize)  is  to  be  attained.  New  designs  for  airborne 
systems  reduce  crewstation  crowding  but  do  nothing  to  reduce 
workload  problems  because  they  still  continue  to  rely  solely  on 
visual  and  motor  task  performance  by  the  operator.  The  research 
discussed  here  is  an  effort  aimed  at  achieving  a  technological 
breakthrough . 

A  technological  breakthrough  is  being  achieved  by  developing 
em  alternative  means  of  communicating  with  the  aircraft  system  by 
the  spoken  word. 

In  general,  voice  systems  allow  the  operator  to  input  data 
or  ask  questions  about  the  status  of  the  system  using  conventional 
speech,  and  to  receive  verbal  status  advisories  or  warnings  as 
well.  This  capability  for  full  "interaction"  by  voice  is  called 
a  voice-interactive  system  (see  figure) . 


moMiMS  AmjCATwet 
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Voice  systems  can  reduce  workload  and  enhance  the  produc¬ 
tivity  of  the  operator  in  a  variety  of  system  applications. 
Laboratory  studies  and  operator  estimates  indicate  that  data  can 
be  entered  into  onboard  computers  two  to  three  times  faster  by 
voice  than  by  manual  keyboard  when  tne  operator  is  performing  a 
control  (hand)  or  visual  task  at  the  same  time.  A  voice  system 
fully  integrated  with  other  weapon  subsystems  can  reduce  time  to 
detect  and  respond  to  an  emergency  by  30%  to  50%,  depending  on 
the  operator's  involvement  with  other  tasks.  Such  time  savings 
during  critical  mission  segments  could  yield  dramatic  returns  in 
improved  mission  performance. 

Demonstration/Validation  Phase  Human  Factors 

In  this  phase,  task  analysis  and  operational  sequence 
analyses  are  conducted  to  determine  what  tasks  are  performed 
in  what  r  i'der  to  accomplish  ea^h  manned  function.  These  analyses 
specify  what  information  needs  to  be  present  and  what  types  , 
of  responses  are  necessary  for  each  task.  These  analyses  also 
specify  what  skills  and  knowledge  are  required  to  perform  the 
task.  The  results  of  the  analyses  are  used  to  develop  station 
arrangement  concepts.  The  stations  can  represent  one  function  of 
the  system  or  a  group  of  similar  tasks  from  all  the  functions . 

The  work- space  is  then,  developed  from  station  arf cingements ., 
There  are  a  number  of  techniques  available  to  maximize  the 
efficiency  of  the  work  space  and  decrease  operating  errors. 

The  console  concept  is  developed  from  the  results  of  the 
work  space  analysis,  the  information  and  response  requirements, 
and  task  clustering.  The  control-display  analyses  deal  with  how 
,  the  required  information  is  to  be  presented  and  what  types  of 
controls  are  best  for  the  responses; 
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The  hvunan  factors  product  of  this  phase  specifies  what 
kinds  and  quality  of  human  performance  are  required,  and  the 
human  engineering  required  for  operators  and  maintainers , 
including  the  information  and  response  needs  at  each  interface. 

Human  Factors  Efforts  and  System 
Development  Activities  During 
Demonstration/Validation 

This  subsection  provides  a  description  of  the  human  factors 
efforts  and  system  development  activities  shown  in  Exhibit  3-8. 
The  descriptions  are  keyed  to  the  numeric  codes  on  the  chart. 

The  product  of  the  Concept  Development  Phase  should  be  input 
directly  to  human  factors  activities  requisite  to  the  Demonstra¬ 
tion/Validation  Phase.  The  product  of  this  phase  contains  two 
components  overall  which  distinguish  two  elements;  (1)  Task 
Analysis,  and  (2)  Human  Engineering  Requirements.  The  results 
of  the  former  serve  as  major  input  to  the  latter  developments. 

(2.1)  Test  and  Evaluation  Master  Plan  (TEMP).  The  TEMP 

is  the  controlling  document  which  derives  test  and  evaluation 
requirements  for  development  test  and  evaluation  and  operational 
test  and  evaluation.  The  TEMP  identifies  decision  criteria  and 
funding  constraints  in  support  of  the  overall  approved  program 
objectives.  '  • 

(Adapted  from  OPNAVINST  5000.42A  and  SECNAVINST  5000. lA) 

(2.2)  Submit  Proposals.  Since  development  of  a  prototype 
system  is  a  major  requirement  of  the  Demonstration  and  Validation 
Phase,  a  request  for  proposal  (RFF)  must  be  prepared,  contractor 
proposals  written  and  submitted,  and  a  winning  contractor (s) 
awarded  the  contract  ;:o  develop  the  prototype.  Requirements  to 
be  issued  in  the  RFP  should  be  obtained  from  the  TEMP,  the  DCP  of 
the  previous  phase,  and  previous  activities'  results. 


(2.3)  Construat  Prototype.  The  purpose  of  a  prototyping 
effort  in  the  Demonstration/Validation  Phase  is  to  confirm 
that  the  technology  is  feasible  and  that  the  design  Concept 
has  military  utility  against  a  stated  military  requirement. 
Prototypes  may  also  be  fabricated  for  competitive  evaluation  to 
select  the  best  approach  for'  further  development.  Human  factors 
activities  relevant  to  the  Demonstration/Validation  Phase  should 
be  conducted  in  association  with  the  development  of  the  prototype 
(Adapted  from  AR  70-1) 

(2.31)  Task  Requirements  Analysis.  Requirements  for  human 
performance  are  generally  gained  through  the  development  of  a 
task  analysis  for  a  specific  system  in  mind.  Most  task  analysis 
requirements  are  derived  through  a  two-step  process  involving: 

(1)  subtask  derivation  and  (2)  skill  and  knowledge  analysis. 
Subtask  derivation  results  in  task  descriptions,  work  designation 
(operator/maintainer/support) ,  task  locus,  and  behavior  and  time/ 
sequence  variables.  Task  skill  and  knowledge  analysis  results  in 
assignment  of  skill  level  to  tasks/subtasks,  military  specialty 
requirements,  and  necessary  and/or  special  knowledge  requirements 
(Adapted  from  VanCott  &  Kinkade,  1972) 

(2.32)  Operational  Sequence  Analysis.  Dynamic  cuialysis  of 
the  operations  environment,  such  as  that  offered  by  operational 
sequence  diagramming  (as  opposed  to  static.  anaU.y sis  such  as  that 
offered  by  task  analysis)*  provides  time-based  data  revealing 
among  other  things  operator  workload  requirements,  performance 
requirements  exceeding  operator  and  equipment  cap2d3ilities ,  and 
sequences  amenable  to  translation  into  training  emd  mission 
scenario  development. 

(2.33)  Maintenance  Requirements  Analysis.  A  maintenance 
task  equipment  requirements'  analysis  is  conducted  to  define 


requisite  maintenance  tasks,  test  equipment  utilized  and 
procedures  for  their  use,  equipment  to  be  maintained,  and 
malfunctions  possible.  This  data  is  used  to  determine  skill 
level  and  knowledge  requirements  for  maintainors  of  system 
components,  and  training  requirements  for  maintenance  (including 
criteria  and  measures  of  performance) .  In  addition,  maintenance 
support  should  be  considered  for  inputs  on  maintenance  of  system 
effectiveness. 

(2.34)  Consider  Maintenatiae  Philoaopnies .  Having  obtained 
detailed  requirements  for  system  maintenance ,  consideration 
should  be  given  to  the  choice  of  maintenance  philosophies  to  be 
implemented.  These  especially  involve  the  input  of  human  factors 
to  agencies  responsible  for  training  and  support  of  maintainers. 
Recommendations  for  training  requirements,  training  devices  and 
simulation  requirements,  and  job  aids  and  manuals  should  also  be 
developed. 

The  previous  discussion  constitutes  what  is  roughly  equiva¬ 
lent  to  the  determination  of  requirements  for  human  performance 
through  hvunan  task  analysis.  This  data  (along  with  data  prepared 
in  previous  stages  of  system  development)  should  be  utilized  in 
the  specification  of  human  factors  engineering  requirements. 

These  requirements  are  then  used  to  support  development  of  the 
prototype  system.  The  ensuing  discussion  details  the  general 
plan  for  this  process. 

(2.35)  Station  Arrangement  Conoepte.  Proceeding  from  a 
general  level  of  detail,  a  preliminary  arrangement  of  personnel 
and  equipment  within  the  workstation  is  made.  Thir  process  is 
generally  based  on  knowledge  of  information  flow  within  the 
station  (as  determined  by  such  a  technique  ai  information  flow 
charting)  as  well  as  knowledge  of  communication  and  operator 
traffic  flows  (as  determinea  in  link  analysis,  for  exaB?>le) . 


(2.36)  Workspace  Concepts.  While  the  station  arrangement 
process  described  above  is  concerned  with  interactive  performance 
of  men  and  machines ,  workspace  concepts  deal  with  environmental 
and  other  associated  aspects  of  the  station  and  other  locations, 
involving  even  men  in  passive  states  with  respect  to  the  system. 
Environmental  effects  deal  with  issues  of  climate  and  habitability. 
Safety/ personnel  mobility,  equipment  space,  and  other  associated 
requirements  must  be  considered  in  relation  to  workspace  conditions 

(2.3?)  Console  Ccncept-i.  The  above  considerations  dealt 
with  crews  and/or  nonspecific  individual  operator  requirements. 
Console  concepts,  on  the  other  hand,  involve  primarily  a  specific 
operator  in  relation  to  specific  duties  and  equipment/components. 
Considerations  that  require  excunination  for  design  implications 
of  individual  operator  consoles  include  operator  visual,  auditory, 
manipulatory,  and  ambulatory  requirements.  Different  standards 
are  applied  to  console  concept  selection  involving  stand-up  and 
sit-down  operators. 

(2.33)  Control-Display  Concepts.  The  final  analysis  in 
relation  to  development  of  personnel  work  stations  is  the  speci¬ 
fication  of  control-display  concepts.  This  paper- and- pencil 
arrangement  analysis  of  controls  and  displays  on  panels  and 
consoles  should  be  based  on  an  analysis  of  operator  utilization 
frequency,  accuracy,  sequence,  etc.  eUTid  the  importance  of  these 
displays  and.  controls  to  controlling  or  monitoring  system  per¬ 
formance.  Guidelines  for  these  purposes  include  general  topic?r 
such  as  the  following: 

1.  Priorities  for  locating  controls  and  displays 

2.  Spacing  between  controls  and  displays 

3.  Grouping  controls  and  displays  to  either  function  or 
sequence 

4.  Sequence  of  operation. 

(Adapted  from  MIL-HDBK-759) 
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The  preceding  discussion  is  clearly  slanted  toward  the 
operations  side  of  a  system.  Analysis  of  the  maintenance  portion 
must  also  proceed  through  a  process  similar  for  operations. 
Concepts  to  be  included  in  this  sort  of  maintenance  analysis  must 
focus  upon  aspects  of  equipment  and  test  equipment  portability, 
equipment  and  workspace  accessibility,  and  procedures  associated 
with  maintenance  actions  (e.g.,  fault  isolation,  correction,  and 
prevention) . 

(2.29'  CcKsidev  Hur.an  Factors  Engineering  (HFE)  Design 
Alternativi.  3 .  The  concepts  derived  above  must  now  be  grouped 
together  to  evolve  alternative  requirements  for  human  factors 
design  wherever  where  is  a  man-machine  interface.  Analysis  at 
this  stage  must  reconcile  potential  crew  interaction  problems 
and  individual  workload  cap2djilities  and  limitations.  Each 
alternative,  as  well  as  the  one  selected  as  optimal,  should  be 
presented  in  the  form  of  drawings,  tabulations,  and  narratives. 

(2.210)  Sinulat ion/Moekup  Evaluations .  Tc  begin  verifica¬ 
tion  of  analyses  performed  up  to  this  point,  as  well  as  to  begin 
HFE  detail  design  based  on  actual  studies  of  the  man-machine 
interface,  t.he  use  of  simulation,  mockups,  and  their  subsequent 
evaluation  roust  be  performed.  These  activities  should  result  in 
determining  the  efficacy  of  the  HFE  design  alternative  recom¬ 
mended,  as  opposed  to  the  remaining  alternatives.  In  addition, 
development  and  refinement  of  specific  HFE  design  parameters 
should  also  proceed  to  the  extent  ox  confirming  the  validity  of 
HFE  design  alternatives— this  can  be  done  to  the  level  of 
drawings,  tabulatio^.s ,  and  narratives,  or  through  providing  data 
pertinent  to  thcix  modification. 

(2.211)  Human  Performance  and  Human  Engineering  Requirement 
As  stated  previously,  the  major  HFE  product  of  the  Dtmcnatration/ 
Validation  Phase  is  the  .^Tpecification  of  human  task  analysis  and 
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human  factors  engineering  requirements.  Since  it  is  a  major 
product,  it  has  been  previously  introduced  as  such  and  will  not 
be  repeated  here. 

(2.4)  Conduct  DT  I/OT  I.  Primary  verification  of  the 
feasibility  of  a  prototype  system  to  achieve  stated  requirements 
is  determined  through  development  test  (DT)  and  operational  test 
(OT) .  In  the  Demonstration/'Validation  Phase  these  are  DT  I/OT  I. 

DT  I  is  conducted  to  demonstrate  that  technical 
risks  have  been  identified  and  that  solutions 
are  in  hand.  Components,  subsystems,  brassboard 
configuration  or  advanced  development  prototypes 
are  examined  to  evaluate  the  potential  application 
of  technology  and  related  design  approaches  prior, 
to  entry  into  full  scale  development. 

(DA  Pam  11-25) 

OT  I  is  conducted  to  determine  military  validity  and  worth 
to  the  user.  OT  I  estimates: 

a.  The  potential  of  the  new  system  in  relocation  to 
existing  capabilities 

b.  The  relative  merits  of  availad>le  competing  prototypes/ 
systems  from  the  aspects  of  military  utility 

c.  The  adequacy  of  the  concepts  for  employment,  support 
ability  organization,  doctrinal,  tactical,  and  training 
requirements,  and  related  critical  issues. 

(Adapted  from  DA  Pam  11-25} 

(2.41)  Conduct  Human  Faetora'  TiS.  Human  factors  test  and 
evaluation  (HP  TfcE)  is  conducted  in  conjunction  with  OT  I.  The 
purpose  of  HP  TiE  for  the  Overall  system  development  is  to 
demon.3trate  that  human  performance  technical  risks  have  been 
identified  along  with  their  solutions.'  HP  TtE  also  attempts  to 
validate  the  human  task  analysis  and  the  human  factors  engineering 
requirements. 
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(2.1::)  '-3^;  Analysis  and  Human  Pastors  Engineering.  Require¬ 

ments  Validated.  The  development  of  validated  task  analysis  and 
human  factors  encjineering  requirements  will  provide  the  next  phas 
of  development  with  valid  human  factors  data  to  firm  up  the 
detail  design  wherever  a  man-machine  interface  is  located. 

(2.c)  I'z'date  PCF.  The  Decision  Coordinating  Paper  (DCP) 
is  updated  to  include  recommendations  for  further  system  devel¬ 
opment  as  well  as  designation  of  preferred ' alternative  designs 
ana  rationales  fcr  such  choices. 

DCARC  IT.  (S)  PARC  IJ.  The  purpose  of  DSARC  II/ 

(S),SARC  II  is  to  evaluate  the  readiness  of  the  system  development 
program  to  enter  full-scale  development.  Reviews  are  conducted 
of  the  DCP,  among  other  documents.  Approval  by  the  DSARC  and 
(SlCAFC  st'ts  the  stage  for  continued  development  of  the  system. 

Content  of  the  Task  Analysis  and 
Human  Engineering  Requirements 
Product 

A  documented  task  analysis  and  statement  of  the  system 
human  engineering  requirements  shall  include  the  following 
considerations: 

As  sumptions : 

Th »  following  items  will  serve  as  input  to  the  process  of 
deter  Pining  humar.  performance  and  human  factors  engineering 
requirements: 

•  MENS 

•  DCP 

•  Products  of  function  allocation. 

Task  analyt'  techniques  will  bo  utilized  to  encompass 
pertinent  aspects  of  operrtions  and  maintenance  for  a 
proposed  system.  Requirements  for  human  factors  engineering 
'  will  also  encompass  op^'rations  and  maintenance. 
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Actions : 


1.  The  principal  product  of  the  human  task  analysis  portion 
of  this  phase  will  be  a  completed  task  analytic  package 
(including  static  and  dynaunic  aspects  for  all  tasks). 
Overall,  the  package  will  provide  the  following  data: 

a.  tasks  and  task  sequences  required  of  operators  and 
maintainers 

b.  actual  equipment  employed 

c.  safety 

d.  maintenance. 

Techniques  utilized  to  derive  these  data  will  include 
procedures  such  as  the  following:  Behavioral  Task 
Analysis,  Opierability /Maintainability  Analysis.  Hazard 
Analysis,  Workload  Analysis,  Task-Equipment  Analysis, 
Operational  Sequence  Diagrams,  and  Link  Analysis. 

2.  The  overall  task  analysis,  including  task  descriptions, 
will  be  presented  in  the  form  of  flow  diagrams,  tabular 
presentations,  ^md  n^u:ratives. 

3.  The  human  task  analysis  will  commence  with  a  summary  of 
gross  tasks.  This  summary  will  demonstrate  the  feasi¬ 
bility  of  achieving  system  performance  r^uirements  as 
well  as  ensuring  that  human  perfbrm2mce  requirements  do 
not  exceed  caped^ilities.  In  addition,  the  effects  upon 
the  following  items  will  be  described: 

a.  manning  level 

b .  equipment  procedures 

c.  requisite  skills  and  training 

d.  communication  requirements  (between  operators  and 
operators  and  the  system) 

e.  logistics  support. 
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The  hiiman  task  analysis  will  specify  tasks  critical  to 
system  performance  as  well  as  evidence  to  support  its 
criticality.  These  tasks  will  include  but  not  be 
limited  to  the  following  data: 

a.  information  requirements  by  operators/maintairiers 
(including  cues  for  task  initiation)  . 

b.  information  available  to  operators/maintainers 

c. .  evaluation  process 

d.  decisi.-ns  reached  after  evaluation 

e.  action  taken 

f.  body  movement  required  by  action  taken 

g.  workspace  envelope  required  by  action  taken 

h.  workspace  available 

i.  location  and  condition  of  work  environment 

j.  frequency  and  tolerance  of  action 

k.  time  base 

l.  feedback,  informing  operators/maintainers  of  the 
adequacy  i)f  action  taken 

m.  tools  and  equipment  required 

n.  number  of  personnel,  specialties,  and  experience 

o.  job  aids  or  references 

p.  communication  required  (including  type) 

q.  hazards  , 

r.  interaction  of  multiple  personnel 

s.  operational  limits  of  personnel  (performance) 

t.  operational  limits  of  machine  and  software. 


The  human  task,  analysis  package  will  provide  the  results 
of  an  opere  ;ility /maintainability  workload  analysis 
(including  i.he  interaction  of  multiple  personnel)  .  The 
operability  analysis  will  detail  the  following: 

a.  design  goal — quantity  and  quality  of  information 
throughput 

b.  predict  expected  quantity  and  quality  of  throughput 
operators  should  expect 

c.  ccxnparison  of  predicted  with  desired  throughput  and 
resolution  of  differences. 

The  maintainadaility  analysis  will  detail  the  following: 

a.  desigtt  g0al--lndlliairf5“tt!e  ''ef  fects  of  >  autonated 
maintenance 

b.  predict  performance  times  for  correction  (including 
identification,  fault  isolation,  and  correction)  of 
-system  malfunctions 

c. .  compare  predicted  maintenance  with  goal  and  resolve 

differences. 

Develop  requirements  for  humem  factors  engineering  by. 
cuialysls  of  effects  of  critical  tasks  upon  system  and 
equifmtent  performance,  cost,  periods  of  peaic  personnel 
workload,  conflict  situations  placing  demands  upon 
personnel  and  equipment  as  well  as  requirements  not 
previously  apparent.  In  addition,  life  support  charac¬ 
teristics  will  be  detailed  covering  but  not  limited  to 
the  following:  noise,  shock  and  vibration,  temperature 
extremes,  atmospheric  contamination,  toxicity,  electric 
shock,  mechanical  hazards,  electromagnetic  and  nuclear 
radiation,  explosion/fire,  pressure  and/or  decompression. 
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This  analysis  will  also  result  in  the  prediction  of  the 
probabilities  for  operator  and  maintainer  error.  Details 
to  be  included  in  the  error  analysis  are: 

a.  identification  of  the  locus  of  errors 

b.  malfunction 

c.  extreme  conditions  and  environments 

d.  effects  of  enemy  action 

e.  recommendations  for  avoidance  cf  design-induced 
error 

f.  rating  of  error  likelihood 

g.  rating  of  error  criticality 

h.  estimate  of  seriousness  of  consequences'  to  personnel 
and/or  equipment;  and  system,  subsystem,  and/or 
component  performance. 

Additional  requirements  for  human  factors . engineering 
involved  with  development  of  procedural  documents , 
personnel  planning,  and  system  testing  will  be  developed. 
This  data  will  be  obtained  from  au>  analysis  resulting 
from  the  compilation  of  task-related  data  into  prelimi¬ 
nary  operator/maintainer  procedurally  oriented  task 
descriptions.  (Especially  important  in  this  regard 
would  be  the  determination  of  system  zmd  personnel 
performance  time  and  accuracy  requirements  to  be  used 
in  system  test  and  evaluation.  A  sequential  analysis 
of  the  operational  sequence  diagram  would  provide  these 
data  on  a  dynamic  basis  suitable  for  this  use.) 


Demonstration/Validation  Phase; 

Example  of  Human  Factors 
Contribution 

The  crewstations  of  an  aircraft  must  be  usable  by  crew¬ 
members  who  vary  widely  in  physical  size.  If  a  significant 
number  of  pilots  cannot  reach  the  critical  controls,  accidents 
and  injuries  will  increase.  Between  15  and  20  aircraft  mishaps 
per  year  have  been  attributed  to  difficulties  in  reach,  st  a  cost 
of  20-30  million  dollars  in  damage.  The  problem  has  become 
serious  enough  that  the  Chief  of  Naval  Operations  recently  asked 
that  aviators  be  matched  with  specific  aircraft  according  to  how 
well  they  "fit"  the  physical  dimensions  of  the  cockpits  and 
controls.  W'ile  this  approach  is  effective  in  reducing  accidents, 
it  limits  tiie  use  of  the  trained  aircrew  population  and  wastes 
valuc^le  training  and  retraining  time.  Pilot/cockpit  size  mis¬ 
matches  can  usually  be  solved  by  early  engineering  design  changes. 
But  to  do  so  requires  that  cockpit  geometry  mismatches  be  detected 
while  the  aircraft  is  still  on  the  drawing  board.  To  that  end, 
the  Navy  heeds  a  method  of  analysis  that  can  compare  and  quantify 
planned  cockpit  geometry  against  the  aircrew  population  at  this 
early  design  stage. 

To  meet  the  need  for  a  method  of  comparison,  NAOC  developed 
the  Crewstation  Assessment  of  Reach  (CAR) .model.  The  CAR  model  ' 
is  based  on  extensive  prior  research  in  industry  and  government. 
This  work  has  resulted  in  sophisticated' cockpit  geometry  models 
which  can  compare  the  physical  dimensions  of  a  specific  operator 
against  the  dimensions  of  a  proposed  crewstation.  The  CAR  model 
uses  a  condensed  version  of  t^ose  earlier  models  to  evaluate  a 
cockpit  design  against  a  statistical  sample  representing  the 
entire  operator  population.  Thus,  CAR  is  able  to  estimate  the 
percentage  of  available  aircrewmen  who  car.  operate  a  proposed 
design  and  the  percentage  who  will  have  difficulty  in  performing 
any  specific  control  action. 
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CAR  is  applied  at  the  earliest  possible  stage  of  design, 
using  the  initial  drawings  as  an  input.  The  model  examines 
hand  and  leg  control  positions,  head/cancpy  clearance ,  and  seat 
movement  required  to  achieve  over-the-nose  vision.  Where  reach 
or  clearance  problems  are  detected,  CAR  identifies  the  controls 
involved.  Because  the  computer  program  is  "interactive,"  the 
researchers  can  immediately  evaluate  alternative  designs.  A 
large  number  of  alternatives  can  be  explored  with  a  minimum 
of  time  and  cost,  and  acceptable  solutions  can  be  identified 
promptly. 

CAR  uses  a  mathematical  model  of  the  human  skeleton,  con¬ 
sisting  of  the  major  body  segments  ("links")  and  the  joints  which 
connect  those  links,  with  all  their  lengths,  limits  of  movement, 
and  variations  in  dimension  within  the  operator  population  (see 
figure) .  It  can  quickly  calculate  how  the  skeletal  model  must 
■  «'  move  to  perform  any  specific  action,  under,  various  conditions  of 

harness  restraint  or  requirements  for  hand  action  (e.g.,  grasp, 

^  touch,  manipulate). 
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CAR  has  been  used  in  the  design  of  three  aircraft:  the 
Light  Airborne  Multi-Purpose  System  (LAMPS)  MK-III  (SH-3) ,  the 
F-18,  and  the  AV-8B.  For  LAMPS  and  the  F-18,  changes  were 
reconunended  and  were  used  in  further  development.  For  the 
AV-»B,  studies  are  still  in  progress  to  correct  some  identified 
problPms. 

Applied  to  the  F-18  preliminary  design,  CAR  revealed  that 
only  10%  of  the  aviator  population  would  be  able  to  use  all 
critical  controls.  The  seat,  stick,  and  emergency  controls  were 
therefore  relocated,  using  CAR  recommendations,  to  accommodate 
nearly  100%  of  aviators.  The  engineering  changes  that  were 
required  included  major  modifications  of  the  aircraft  structure. 

CAR  has  been  adopted  for  use  elsewhere  in  the  government 
euid  in  industry.  Within  the  government,  it  has  been  modified  by 
NASA  for  Space  Shuttle  design.  In  industry,  it  has  been  used 
in-house  by  McDonnell  Douglas,  Northrup,  Sikorsky,  the  Clark 
Equipment  Company,  and  IBM. 

Earlier  methods  of  analyzing i cockpit  geometry  required 
laborious  manual  procedures,  or  else  computer  models  not  suitable 
for  use  in  early  stages  of  design,  The  results  were  often 
expensive,  late,  imprecise,  and  h,ird  to  convey  to  design  engineers 
At  worst,  problems  remained  undet(jcted  until  the  aircraft  were  in 
service,  and,  then  often  surfaced  ,is  accident  data.  CAR  improves 
the  accuracy  of  cinalysis  while  reducing  the  time  required  from 
more  than  two  weeks  to  less  than  i  day,  with  a  90%  decrease  in 
cost.  CA.R  can  be  applied  earlier  in  the  design  cycle  them  ever 
before,  and  used  interactively  to  find  engineering  answers  and 
test  them  ediead  of  time.  It  will  produce  aircraft  which  are  more 
mission-effective  because  they  are  better  fitted  to  the  aircrew. 
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For  example,  when  used  to  guide  design  of  the  .F-18 ,  CAR  made 
it  possible  to  correct  on  paper  design  deficiencies  that  would 
have  cost  millions  to  change  if  not  detected  until  construction 
began.  Savings  through  avoiding  lost  training  time  for  pilots 
who  could  not  have  safely  used  the  initial  design  are  estimated 
at  10  to  40  million  dollars  per  year. 

Full-Scale  Development  Phase  H,uman  Factors 

This  phase  should  result  in  a  firm  and  detailed  man-machine 
interface  design.  At  the  start  of  the  phase,  various  man-machine 
combixiaticns  should  be  considered  that  would  satisfy  the  human 
performance  and  human  factors  engineering  requirements.  The 
testing  of  these  combinations  by  simulation  or  mockups  should 
identify  the  most  effective  combinations.  Simulation  trials  are 
a  good  method  for  pinpointing  peak  personnel  and  equipment  work¬ 
loads,  detecting  probable  human  errors,  identifying  inefficient 
interfaces,  and  determining  if  the  design  is  appropriate  for  the 
intended  user. 

The  equipment  must  be  designed  to  meet  the  physical  and 
cognitive  needs  of  the  intended  user.  Physically,  the  equipment 
must  be  designed  to  permit  effective  movement,  effective  use  of 
knobs  and  controls,  transporting  of  goods,  effective  use  of  arms 
and  legs,  or  whatever  the  tasks  call  for.  Cognitively,  the  human 
factors  engineer  is  responsible  for  recommending  designs  that 
are  neither  too  difficult,  nor  simplistic,  but  that,  rather, 
assign  the  proper  cunount  of  cognitive  workload  to  different 
kinds  of  users.  To  have  an  effective  system,  the  design  must 
differ  for  different  types  of  users.  One  design  would  be  appro¬ 
priate  if  a  position  is  expected  to  have  high  turnover  in  short 
periods  of  time  and  the  user  is  to  be  minimally  trained  and  to 
possess  few  skills,  while  another  design  would  be  appropriate  for 
a  position  with  less  turnover  and  occupied  by  a  user  with  better 
training,  skills,  knowledge,  and  ability  to  make  complex  decisions 
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The  man-machine  tests  and  evaluations  should  cover  every 
detail  of  the  design,  from  such  things  as  effectiveness  of  a 
type  of  information  displayed  at  various  times  to  control-display 
compatibility,  spacing  of  controls,  shape  of  controls,  sequence 
of  controls,  anthropometry,  and  in  general,  all  of  those  areas 
in  MIL-STDrl472B  and  MIL-H-46855B.  The  HF  T&E  in  this  phase 
should  result  in  a  relicdsle  man-machine  design. 

Human.  Factors  Efforts  and  System 
Development  Activities  During  . 

Full-Scale  Development 

The  major  human  factors  products  of  the  Demonstration/ 
Validation  Phase,  task  analysis,  and  human ' engineering  require¬ 
ments  serve  as  the  basic  data  for  the  detailed  human  factors 
design  of  the  system  wherever  a  man-machine  interface  occurs. 

This  subsection  provides  a  description  of  the  human . factors 
efforts  and  system  development  activities  shown  in  Exhibit  3-9. 
The  descriptions  are  keyed  to  the  numeric  codes  on  the  chart. 

(5.1)  Submit  PropoaalB,  It  is  possible  that  in  full-scale 
development  a  different,  contractor  may  be  selected  than  the 
contractor  employed  in  the  Demonstration  and  Validation  Phase. 

In  addition,  it  is  also  possible  (albeit  unlikely)  to  continue 
with  competitive  developments.  For  these  reasons  (although  they 
are  somewhat  rare  in  occurrence)  the  complete  cycle  involving 
RFPs,  proposals,  and  contract  award  is  repeated. 

3.2  Conatruat  Prototype( a) .  When  development  prototypes 
are  fabricated  in  full-scale  development,  the  intent  is  to  assure 
that  the  engineering  problems  have  been  solved  and  to  permit 
thorough  evaluation  of  the  system.  (This  occurs  prior  to  a 
commitment  to  full-scale  production  or  simultameously  with  low- 
rate  initial  production. ) 

(AR  70-1) 


Exhibit  3-9 


Specific  Human  Factors  Efforts  During  the  Full-Scah’  Dcvelopnient  Phase  (3.) 


ArmOVE  DESIGN 


It  has  traditionally  been  during  this  stage  of  development 
(prototyping  in  full-scale  development)  that  the  bulk  of  hxjman 
factors  R&D  has  occurred. 

(2.21)  Consider  HFE  Design  Alternatives.  Before  entering 
full-scale  development,  a  decision  is  required  whether  to 

accept  or  modify  the  prototype  system  built  and  tested  previously  • 
in  the  Demonstration/Validation  Phase.  For  HFE  proper,  this 
encompasses  the  man-machine  interface.  This  is  necessary  when 

I 

any  or  all  of  the  following  events  occur: 

1.  Contractor  awarded  full-scale  development  is  different 
from  contractor  during  Demonstration/Validation  Phase. 

2.  Deficiencies  identified  through  the  HFE  portion  of  OT  I 
require  modification  to  the  system  (this  may  also  include 
DT  I/OT  I  findings  at  large  as  well) . 

3.  Design  requirements  change  development  of  hardware  and 
software  components  (e.g.,  to  take  advantage  of  newly 
breaking  technologies) ,  thus  forcing  HFE  to  keep  abreast 
rf  development. 

Based  on  events  occurring  as  illustrated  above,  HFE  will  provide 
design  recommendatipns. 

(2.22)  Simulation/Mo akup  Evaluations.  Should  hardware  and/ 
or  software  design  requirements  be  modified  resulting  in  changes 
to  the  man-machine  interface,  new  studies  cuid  analyses  involving 
simulation  and/or  mockups  may  become  necesssury  to  evaluate  che 
effects  of  change  upon  personnel  ( opera  tor  s/maintainer.,}  workload, 
and  assigned  activities.  Analysis  may  be  reqxiired  especially  when 
the  effects  of  design  changes  are  unknown. 

(2.22)  Detailed  Man-Maohine  Interface  Design.  Final  HFE 
design  requirements  should  be  prepared  in  the  following  formats: 
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drawings,  tabulations,  and  narratives.  This  will  facilitate  their 
implementation  in  detail  design  of  the  full-scale  development 
prototype.  HFE  requirements  include  human  engineering  principles 
and  criteria  which  offer  assurance'  that  the  final  product  can  be 
efficiently,  reliably,  and  safely  operated  and  maintained. 
Relevamt  locations  for  the  application  and  human  factors 
engineering  include  the  work  environment,  crewstation,  and 
facilities  being  designed  for  the  system. 

(Portions  from  MIL-H-46855) 

The  product  of  this  phase  is  the  design  of  an  optimal  man- 
machine  iifterface.  It  is  discussed  elsewhere  amd  will  not  be 
repeated  here. 

(2.2)  Conduct  DT  II/OT  II,  Developmental  Test  II  and 
Operational  Test  II  (DT  II/OT  II)  are  required  to  determine 
whether  or  not  the  full-scale  development  prototype  is  ready 
for  production. 

DT  II  ensures  that  engineering  is  reasonably  complete,  that 
all  significant  design  problems  have  been  identified,  and  that 
solutions  are  in  hand. 

OT  II  provides  a  valid  estimate  of  ej^ected  system  opera¬ 
tional  effectiveness  and  suitability  as  determined  through  tests 
involving  the  aid  of  operational  and  support  personnel  of  the 
type  emd  qualifications  of  those  who  are  expected  to  use  and 
maintain  the  system  when  deployed. 

(Adapted  from  AR  70-10) 

t 

(2.2i)  Conduct  HP  TiE.  Conjointly  with  OT  II,  Human  Factors 
test  and  Evaluation  of  the  full-scale  development  system  will  be 
conducted  to: 
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1.  Assure  fulfillment  of  the  applicable  requirements 

2.  Demonstrate  conformance  to  HFE  design  criteria 

,3.  Determine  whether  undesirable  design  or  procedural 
features  have  been  introduced. 

(Adapted  from  MIL-H-46855) 

(3.22)  Detailed  Han-Machine  Interface  Design  Validated, 
The  results  of  an  HF  TiE  should  be  validation  and  verification 
of  design  requirements  to  provide  an  optimal  man-machine 
interface  design. 

(2.4)  Update.  DCP.  The  Decision  Coordinating  ®aper  is 
updated  to  include  a  current  evaluation  of  the  system.  The 
decision  to  proceed  into  full  production  must  be  based  on  this 
DCP. 


(2.S)  DSAEC  III/(S)SARC  III:  Defense  System  Acquisition 
Review  Council  III/ (Service)  System  Acquisition  Review  Council 
III  purposes  are  to  recononend  to  the  Secretary  of  Defense 
approval  of  production  (or,  possibly,  low-rate  initial  production) 
of  a  system.  The  DCP,  among  other  documents,  is  reviewed  during 
this  process. 

(DA  Pam  11-25) 

Content  of  the  Optimal  Man-Machine 
Interface  Design 

The  optimal  man-machine  interface  design  recommendations 
should  include  the  following  considerations: 

Assumptions : 

The  following  items  will  be  regarded  as  inputs  to  the  human 
factors  engineering  design  of  the  man-machine  interface: 


•  Design  criteria  documents  (e.g.,  MrL-STD-14721 

•  Performance  specifications 

•  Drawings  and  data  (e.g.,  functional  flow  diagrauns, 
schematic  block  diagrams,  interface  control  drawings, 
overall  layout  drawings) 

•  Human  factors  engineering  input  (e.g.,  task  amalysis) 
converted  to  detail  equipment  design  features. 

The  following  processed  are  considered  characteristic  of 

this  phase  of  system  development: 

•  Human  factors  engineering  studies,  experiments,  and 
laboratory  tests  (to  resolve  human  factors  and  life 
support  issues) 

•  Mockups  and  models 

•  Dynamic  simulation  (necessary  for  detail  design  of 
equipment  requiring  critical  human  performance) 

•  Human  factors  engineering  contributions  to  detail 
design 

•  Human  factors  engineering  contributions  to  roampower, 
personnel,  and  training  issues  as  a  consequence  of 
detail  design 

•  Human  factors  contributions  to  test  and  evaluation. 

Actions : 

1.  Effects  of  the  working  environment,  including 
habitability  and  operability,  will  be  preue^ted. 

These  effects  will  cover  the  following  artas:  work 
environment,  crew  stations,  and  facilities.  The 
incorporation  of  hvunan  factors  into  the  detail 
design  of  the  above  will  be  demonstrated  by  presenting 
detail  design  drawings,  specifications,  etc.  for  the 
following  three  conditions normal,  unusual,  emergency 
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Topics  to  receive  coverage  will  include  at  least  the 
following: 

a.  atmospheric  conditions 

b.  weather  and  climate 

c.  range  of  accelerative  forces 

d.  acoustic  noise,  shock,  and  vibration 

e.  disorientation 

f.  accessibility 

g.  adequate  visual,  auditory,  and  physical  links 

h.  adequate  non-workspace  areas 

i.  psychophysical  stress 

j .  fatigue 

k.  clothing  and  personal  equipxnent 

l.  equipment  handling 

m.  chemical,  biological,  electrical,  electromagnetic, 
toxicological,  and  radiological  effects 

n.  illumination 

o.  sustenance,  storage,  and  refuse 

p.  safety  protection. 

The  incorporation  of  human  factors  in  detail  design  of 
the  crewstation  layout/arrangement  and  of  equipment 
having  an  operator/maintainer  interface  will  be  demon¬ 
strated.  This  will  include  'the  presentation  of  drawings 
illustrating  the  inclusion  of  human  factors;  for 
ex^unple:  panel  layout  drawings,  communication  system 
drawings,  overall  layout  drawings,  and  control  drawings. 
The  following  additional  items  will  be  requisite  to  the 
demonstration  of  the  inclusion  of  human  factors  in  system 
detail  design: 

*  -  '  •  . 
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a.  ingress  and  egress  to  workspace  and  facilities 

b.  a  list  of  panels,  racks,  controls,  displays,  and 
indicators  existing  at  the  time  of  documentation 
which  have  received  human  factors  approval 

c.  rationale  of  human  factors  layout/arrangement, 
detail  design  of  crew  station(s),  and  any  equipment 
having  an  operator/maintainer  interface 

d.  a  list  of  considerations  used  to  arrive  at  design 
decisions:  results  of  studies,  requirements  based 
on  task  analysis,  mock-up  tests,  mock-up  based 
decisions,  and  simulations 

e.  a  list  and  explanation  for  deviations  from  human 
factors  or  design  requirements  to  the  man-machine 
interface 

f.  sketches,  drawings,  and  photographs  of  required  or 
anticipated  panel  and  rack  arrangements  or  new 
designs/design  modifications 

g.  drawings  or  photographs  of  each  crews tation  design 
showing  locations  of  all  crewstation  panels  in 
relation  to  seat/operator  position. 

The  inclusion  of  human  factors  in  design  considerations 
involving  the  Interaction  of  maintenance  technicians  with 
their  respective  equipment  will  be  demonstrated.  In 
general,  this  will  depict  the  following  steps/stages: 

a.  recognition  of  malfunctions  (displays) 

b.  isolation  of  malfunctions  (troubleshooting) 

c.  fault  correction  (access,  removal  and  replacement, 
repair) . 


A  human  factors  maintainability/accessibility  design 
analysis  will  be  presented  to  include  at  least  the 
following; 

a.  preliminary  drawings,  sketches,  or  photographs 
showing  each  equipment  and  location  in  relation  to 
surrounding  equipment,  passageways,  and  structures 
(this  includes  ancillary  equipment  also) 

b.  rationale  of  human  factors  design  of  each  item 
requiring  maintenance  as  well  as  presentation  of 
decisions  used  to  drive  the  decision  process 
(e.g.,  MIL-STD-1472 ,  results  of  studies,  simulation 
mockups ) 

c.  incorporation  of  maintenance  task  analysis 

d.  descriptions  to  include  but  not  be  limited  to  the 
following: 

•  physical  size,  purpose  of  support,  and  test 
equipment  required  for  maintenance 

•  maintenance  procedures 

•  relation  between  accessibility  and  failure  rate, 
service  frequency,  calibration  frequency,  emd 
requirements  for  rapid  maintenance 

•  methods  used  to  determine  accessibility  for 
maintenance 

a  anticipated  maintenance  and  accessibility 
problem  areas. 

4.  Best  available  data  on  equipment  operating  procedures, 
operational  sequence  diagrams,  and  task  analysis  will 
be  provided  to  organizations  responsible  for  manpower 
development. 
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5.  A  human  factors  test  and  evaluation  plan  will  be 

prepared  to  cover  the  following  general  concepts: 

a.  fulfillment  of  human  factors  requirements 
V  b.  conformance  to  human  factors  design  criteria 

c.  quantitative  measures  of  system  performance 

d.  detection  of  undesirable  design  or  procedural 
features. 

Full-Scale  Development  Phase: 

Two  Examples  of  Human  Factors 
Contributions 

The  ex^unple  that  follows  is  actually  a  case  of  system 
modification  rather  than  system  development,  but  the  human 
factors  impact  is  conceptually  the  same. 

The  Strategic  Air  Command  (SAC)  is  continually  considering 
proposed  new  hardware  or  modifications  to  existing  hardware  for 
improving  offensive  and  defensive  avionics  equipment  in  the  B-52 
fleet.  These  new  subsystems  are  expensive  and  must  be  evaluated 
with  respect  to  the  degree  of  additional  combat  crew  effectiveness 
that  can  be  realized.  In  short,  the  new  equipment  must  be 
operable  in  the  B-52  mission  anvironment  by  the  current  population 
of'  electronic  warfare  officers  amd  must  result  in  improved  mission 
perfor^nce. 

The  Strategic  Avionics  Crew  Station  Design  Evaluation 
Facility  (SACDEF) ,  developed  by  the  Humem  Engineering  Division 
at  Wright-Patterson  Air  Force  Base,  is  on-line  to  provide  the 
effective  selection  of  offensive  and  defensive  avionics  equipment 
and  integration  for  B-52  improvements.  This  capability  involves 
the  quantification  of  Strategic  Air  Command  crewmember  actions  in 
conducting  simulated  Single  Integrated  Operations  Plan  (SIOP) 
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missions.  It  uses  computer- integrated  electronic  warfare  and 
bombing/navigation  systems  simulators  to  evaluate  performance 
improvements  obtained  by  means  of  proposed  new  equipments  and 
crew  station  reconfigurations. 

Trained  crewmembers'  performance  is  monitored  emd  statis¬ 
tical  emalyses  of  combat  crew  performemce  with  the  proposed 
hardware  are  provided  to  SAC.  Recommendatiorfs  made  by  the  human 
factors  specialists  have  been  well  received  (87  of  93  original 
recommendations  were  adopted  in  the  Phase  VI  update  of  the  B-52 
fleet) .  Further,  the  Strategic  Air  Command  has  adopted  the 
policy  that  no  new  hardware  will  be  installed  on  the  B-52  fleet 
until  their  trained  crewmembers  have  participated  in  the  evalu¬ 
ation  studies  conducted  at  Wright- Patterson  AFB.  The  results  of 
this  capability  are  not  only  used  directly  by  the  using  command, 
but  also  provide  technology  advances  in  systems  effectiveness 
modeling  and  simulation  that  will  enhance  man-machine  integration 
"try-before-buy”  evaluation  of  new  weapon  systems. 

The  Aerospace  Medical  Research  Lcd>oratory  is  now  developing 
a  coroparcUale  test  and  evaluation  capability  for  navigator/radar 
navigator  functions  in  conjxinction  with  the  proposed  update  of 
the  B-52  -wfonics  systems  in  support  of  SAC  RQC  75-6. 

A  final  example  follows  which  fully  Illustrates  the  intact 
of  human  factors  R&D  upon  design  and  development  of  military* 
systems  in  the  Full-Scale  Development  Phase  (Gartner  et.  al. t 
1958) . 

» 

Good  human  engineering  of  equipment  controls  and  displays 
has  long  been  a  hallmark  of  human  factors  R&D.  This  is  due  to 
the  fact  that  well  human  engineered  equipment  design  can  have  a 
dramatic  effect  upon  the  operability  of  systems.  Rarely,  though. 
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has  there  been  an  opportunity  to  compare  a  "pre-human  engineered" 
design  of  an  item  of  equipment  with  a  human  engineered  version  in 
order  to  substantiate  claims  as  to  the  value  of  this  particular 
human  factors  technology.  One  such  opportunity  did  arise, 
however,  when  an  empirical  experimental  investigation  between 
two  alternative  designs  of  test  equipment  for  complex  naval  mines 
was  performed.  This  project,  which  involved  development  of 
design  recommehdations ,  detail  design,  fabrication,  experimental 
investigation,  and  evaluation  of  test  equipment  items,  was 
embedded  within  a  larger  program  to  offer  humcm  engineering 
support  to  test  equipment  designs  as  well  as  development  of  a 
human  engineering  design  guidebook  for  engineers. 

As  outlined  above,  the  following  approach  was  taken: 

Two  test  equipment  items  were  fabricated  according  to  human 
engineering  recommendations  aiid  the  same  two  items  were  feibri- 
cated  according  to  their  original  designs.  The  original  design 
test  equipment  and  the  human  engineered  test  equipment  were  then 
empirically  compared  against  each  other  with  respect  to  criteria 
of  time  and  error.  Results  indicated  that  successful  improve¬ 
ments  in  performance  (i.e.,  use)  of  the  human  engineered  test 
equipment  occurred  with  regard  to  reductions  both  in  time  to 
task  completion  and  reduced  error  likelihood.  With  practice, 
performance  on  both  test  equipment  designs  converged.  However, 
since  .it  was  known  that  utilization  of  the  test  equipment  was 
to  be  too  infrequent  for  users  to  sustain  learning,  the  human 
engineered  version  was  considered  necessary.  Furthermore,  it 
was  also  felt  that  practice  effects  would  operate  for  either 
design  version,  with  the  expectation  of  greater  influence  on 
operators  using  the  human  engineered  equipment. 

This  research  demonstrates  the  value  of  human  factors  R&O  in 
Navy  mine  test  set  design.  Implementation  of  human  factors 
design  recommendations  (such  as  were  developed  for  the  test  set 
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equipment  and  documented  in  a  guidebook  for  test  equipment 
development  and  design)  in  a  systematic  and  standardized  manner 
could  lead  to  greater  user  performance  differences  between  the 
human  engineered  and  non-human  engineered  designs.  This  may  be 
due  to  overall  familiarity  with  characteristic  arrangements  of 
controls  and  displays.  Standardization  procedures  in  humem 
engineering  design  which  utilize  known  population  stereotypes 
with  regard  to  user  expectations  could  result  in  dramatic 
reductions  in  time  to  task  completion  and  human  error.  These 
goals  are  crucial  to  design  of  an  optimal  man-machine  interface. 

Production  and  Deployment  Phase  Human  Factors 

If  human  factors  have  been  properly  thought  out  and  executed, 
the  system  should  be  ready  for  production  and  deployment  after 
Milestone  3.  However,  there  are  provisions  for  additional 
engineering  and  human  factors  testing  (DT/OT  III)  if  the  tests 
and  evaluations  in  DT/OT  II  indicate  problems.  The  additional 
testing  is  conducted  on  the  first  few  systems  in  the  initial  low- 
rate  production.  When  additional  testing  indicates  the  problems 
have  been  corrected,  the  system  goes  into  full-scale  production. 

If  any  problems  arise  or  if  improvements  seem  prudent  after 
the  system  iS  fielded,  the  system  acquisition  model  provides  for 
additional  testing  to  recommend  design  changes.  A  good  human 
factors  plan  will  keep  these  costly  design  chemges  to  a  minimum. 

The  scope  of  the  present  project  is  strictly  with  the  system 
development  phases  of  the  acquisition  process  and  is  not  concerned 
with  production  and  deployment.  However,  to  provide  some  closure 
to  the  process,  a  brief  description  of  production  and  deployment 
follows.  No  humem  factors  products  are  defined,  and  no  examples 
are  Included. 


Human  Factors ’ Efforts  and  System 
Development  Activities  During 
Production  and  Deployment 

This  subsection  provides  brief  descriptions  of  the  general 
human  factors  efforts  and  system  production  and  deployment 
activities  shown  in  Exhibit  3-10.  The  descriptions  are  keyed  to 
the  numeric  codes  on  the  chaurt. 

•  (4.1)  Initial  Produation.  When  it  is  decided  to  enter 
production  of  a  system,  initial  production  items  are  generally 
used  for  production  tests  and  follow-on  evaluations  as  necessary. 
Generally,  production  is  not  suppressed  to  await  completion  of 
follow-on  evaluation  (nor  for  that  matter  does  deployment  await 
conclusion  of  this  evaluation) . 

(Adapted  from  AR  1000-1) 

(4.2)  Conduct  DT  III/OT  III.  Development  Test  III/ 
Operational  Test  III  are  conducted  to  determine  if  production 
units  have  the  cap2d}ilities  demonstrated  in  prototypes  and  are 
operationally  suitable  and  effective. 

DT  III  is  conducted  on  production  prototypes  or  production 
items  delivered  frorai  either  an  initial  or  a  pilot  production  run. 
The  purpose  is  to  verify  their  adequacy  and  quality  when  they 
are  ptoduced  in  quantity  and  according  to  production  contract 
specifications,  using  quantity  production  processes^  This  test 
determines  whether  or  not  the  tremsition  from  tai  engineering 
development  prototype  to  a  production  item  has  been  made 
successfully. 

OT  III  is  normally  a  test  of  initial  production  and  has  the 
fundamental  purpose  of  providing  data  on  the  item  or  system  in 
order  to  estimate  its  operational  suit£d}ility,  verifying  that  all 
testable  critical  issues  have  been  resolved,  and  determining  that 
all  benefits  and  burdens  of  the  item  or  system  are  identified. 
(Adapted  from  AR  70-10) 
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Exhibit  3-10 

‘cific  Human  Factors  Efforts  During  Production  and  Deployment  (4.) 
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Additional  testing  is  implemented  where  required  to  resolve 
(hopefully)  residual  problems. 

(4.21)  Study  Problems.  Any  HFE  problems  identified 
following  Milestone  3  are  studied  to  determine  means  to  alleviate 
them.  In  considering  redesign,  an  analysis  of  the  loss  of 
production  time,  increased  costs  due  to  redesign  effort,  and 
production  costs  must  be  made  in  order  to  realistically  determine 
what,  hvunan  factors  alternatives  are  feasible.  Often,  consider¬ 
ation  is  given  to  increased  training  and/or  personnel  with  higher 
skill  levels  than  was  previously  decided. 

(4.22)  Conduct  HF  T&E.  Additional  humeui  factors  test 
and  evaluation  (HF  T&E)  may  be  ecessary  to  (1) .  determine  the 
efficacy  of  the  proposed  change  ..  (2)  determine  how  well  a 
specific  change  has  improved  operation/maintenamce  of  the  system. 

(4.23)  New  Configuration.  A  new  man-machine  interface 
is  configured  as  a  result  of  human  factors  design  changes. 
Personnel  and  training  requirements  as  well  as  system  opera¬ 
bility  are  often  affected  as  a  result  of  such  changes,  and  may 
necessitate  further  investigation. 

(4.3)  Full  Froduotion.  Full-scale  production  will  proceed 
following  approval  based  on  findings  of  DT  III/OT  III. 

(4.4)  Deployment  Preparation.  Deployment  of  systems  to 
the  field  includes  not  only  delivery  and.  set-up  of  the  new 
system.  It  also  requires  fulfillment  of  requirements  found  in 

an  initial  operational  capability,  such  as:  user  unit  is  equipped 
with  production  items  that  are  deemed  suictible,  with  unit 
personnel  that  are  adequately  trained  to  operate,  care  for,  and 
maintain  the  item,  and  the  \mit  has  the  capability  to  perform  its 
assigned  mission.  r 

(Adapted  from  TRADOC  Reg.  600-4) 


3-74 


(4.5)  Retrofit/Improvemente.  Based  on  pro*:  lems  identified 
in  actual  use.-  or  change  in  doctrine ,  threat  or  mission  product 
improvements  and/or  retrofit  programs  may  be  needed  to  resolve 
them.  This  also  requires  the  cognizance  of,  and  often  the  analysis 
by,  HFE  personnel  to  ensure  that  personnel  and  training  require¬ 
ments  are  covered  as  well  as  that  system  man-machine  interface 
optimizatioti  is  maintained. 

(4.51)  Study  Problems.  See  4.21, 

(4.52)  Conduct  HF  T&E.  See  4.22. 

(4.53)  Sew  Configuration.  See  4.23. 
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CHAPTER  4 


HUMAN  FACTORS  RDT&E  IN  THE  TECHNOLOGY  BASE 
AND  THE  CONTRIBUTION  TO  SYSTEM  DEVELOPMENT 


There  has  occasionally  been  some  concern  expressed  that 
Training  and  Personnel  System  Technology  RDT&E  is  not  fully 
justified  as  being  necessary  to  support  specific  system 
development.  This  concern  is  not  directed  at  human  factors 
in  particular,  but  encompasses  manpower  and  personnel,  education, 
and  training,  and  simulation  and  training  devices.  Establishment 
of  a  clear  correlation  between  the  funding  and  performance  of 
technology  base  R&D  and  its  utilization  in  specific  systems 
development  is  difficult,  and  is  at  any  rate  beyond  the  scope 
of  this  project.  However,  in  the  human  factors  category  it  is 
possible  to  discuss  the  potential  for  technology  base  RDT&E  to 
support  specific  system  development  by  relating  the  technology 
base  R&D  to  the  principal  human  factors  products  of  each  phase 
of  system  development.  In  other  words,  the  technology  base  R&D 
in  human  factors  should  also  be  identified  with: 

1.  Determining  the  role  of  man 

2.  Allocation  of  functions  to  man 

3..  Task  analysis  and  human  engineering  requirements 

4.  Design  of  optimal  man-machine  interfaces. 

By  combining  the  system  phase/human  factors  products  with  the 
DOD  classifications  for  human  factors  into  a  matrix,  areas  of 
oppdrtunity  for  human  factors  and  system  development  have  been 
characterized.  This  matrix  is  shown  as  Exhibit  4-1.  Exhibit  4-2 
is  included  to  remind  the  reader  what  kinds  of  research  are 
included  in  each  classification.  Each  cell  of  this  matrix 
represents  an  area  of  opportunity  for  human  factors  to  eventually 
contribute  to  a  specific  system  development. 


MAN-MACHINE  INTERFACE  DESIGN 


Exhibit  4-2 

The  Classification  System  for  Human  Factors  R&D 

(Taken  fiom  the  TCP  for  FY 1979,  tee  Erickson,  Miles,  <£.  Secrist,  1978) 


Areu 

Examples 

Human  related 

Physical  characteristics 

Sensory  capabilities 

Information  processing 

Forecasting  job  requirements 

Measures  of  effectiveness 

Human-machine  related 

Plight  instrumentation 

(subsystem  oriented) 

Equipment  layout 

Maintenance 

Workload  assessment 

Human-machine  mission  related 

Strategic  offense  and  defense  command  &  control 
Tactical  offense  and  defense  command  &  control 
Command  &  control 

Measures  of  system  effectiveness  with  inputs 

Several  things  will  influence  whether  the  technology  base  , 
R&O  in  human  factors  does  in  fact  contribute  to  the  development 
of  a  weapon  system  (or  to  the  decision  npt  to  develop  a  system). 
We  Shall  briefly  address  only  two:  '  (1)  R&D  funding  categories r 
and  t2)  the  auditing  method  problem.  These  two  were  selected 
because  all  human  factors  (in  the  technology  base)  is  supported 
by  a  particular  category  of  funds,  and  because  a  cause-effect 
relationship  between  research  and  utilization  is  a  very  difficult 
thing  to  establish  and  measure. 


R&D  Funding  Categories 


Funding  for  all  technology  base  areas  in  DOD  is  provided 
within  program  elements  in  the  budget  which  provide  for  fvinds 
generally  categorized  as  basic  research  (6,1  funds),  exploratory 
development  (6; 2  funds),  or  advanced  development  (6.3  funds). 
bOD  defines  these  funding  categories  as  follows; 

6.1  Basic  Research — scientific  study  and  experi¬ 
mentation  directed  toward  increasing  knowledge 
and  understanding  in  those  fields  of  the  sciences 

,  related  to  long-term  national  security  needs. 

It  provides  fundamental  knowledge  for  the  solution 
of  identified  military  problems  and  furnishes  part 
of  the  base  for  subsequent  exploratory  cuid  advanced 
developments  in  defense- related  technologies  euid 
new  or  improved  military  functional  capabilitfies, 

6.2  Exploratory  Development— includes  all  effort 
directed  toward  the  solution  of  broadly  defined 
problems,  short  of  major  development  programs,' 
with  a  view  to  developing  and  evaluating  technical 
feasibility. 

6.3  Advanced , Development — includes  all  projects 
that  have  moved  into  the  development  of  hardware 
for  test.  .The  prime  result  of  this  type  of  . 
effort  is. proof  of  design  concept  rather  than 
the  development  of  hardware  for  service  use. 

Projects  in  this  category  have  a  potential  military 
application. 


Advanfced  development  is  divided  into  two  subcategorles : 
nonsystem  advanced  development  (6.3A),  addressing  technological 
option  uncertainties;  and  systems  advanced  development  (6.3B)., 
which  is  the  design  of  items  (usually  heurdware)  for  test  or 
experimentation. 


It  is  very  difficult  to  determine  a  clear  point  at  which  a 
research  and  development  activity  moves  from  6.1  to  6.2  to  6.3. 
The  distinction  is  often  somewhat  arbitrary.  Some  research 


t 

JJ. 


4-4 


activities  are  in  fact  supported  by  more  than  one  funding 
category.  Additionally,  the  individual  services  have  different 
management  organizations  and  systems  for  administering  technology 
base  funds.  Finally,  funding  categories  overlap  in  time;  that 
is,  6.1  does  not  end  abruptly  and  6.2  begin,  but  rather  a  project 
funded  under  6.1  may  overlap  in  time  with  the  same  project  being 
supported  by  6.2  funds.  In  short,  it  is  not  possible  to  precisely 
and  consistently  identify  funding  categories  and  research  progress 
for  human  factors  projects  in  the  technology  base. 

Tracking  Research  to  Utilization 

There  is  no  established  method  for  tracking  or  auditing 
the  results  of  technology  base  research  to  eventual  utilization. 
This  is  true  not  only  in  the  human  factors  or  the  training  and 
personnel  systems  technology  areas,  but  in  other  technologies 
as  N*ell.  Two  studies  will  be  briefly  discussed  to  illustrate 
this  problem. 

The  first  study  was  conducted  by  the  General  Accounting 
Office  (GAO)  in  1977  and  was  a  review  of  human  resources  research 
and  development  (now  the  training  and  personnel  systems  technology 
area)  in  the  Department  of  Defense.  The  excerpt  below  was  taken 
directly  from  the  digest  of  that  study. 

Eight  Defense  research  and  development  organizations 
identified  374  reports  on  human  resources  research 
and  development  published  during  calendar  years  1973 
through  1975  which  were  Intended  to  support  changes 
to : 

—  regulations,  orders, .doctrines,  policies,  or 
manuals; 

—  courses  of  instruction  or  training  programs;  or 

—  equipment. 
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GAO  then  asked  the  intended  users  how  the  results 
v’ere  used  and  any  reasons  for  not  using  them  and 
found  that: 

--  56  percent  of  the  reports  were  used, 

--  38  percent  were  not  used,  and 

--  6  percent  were  being  considered  for  possible 
use. 

The  GAO  concluded  that  the  Department  of  Defense  could 
improve  utilization  by  more  effective  management.  The  authors 
of  the  present  report  find  no  quarrel  with  this  conclusion,  but 
do  have  some  serious  concerns  about  the  method  used  to  arrive 
at  the  results.  Those  concerns  will  not  be  pursued  here. 
Nevertheless,  a  few  points  can  be  made  about  the  review  and 
results  as  reported  in  the  excerpt  above.  First  of  all,  if 
indeed  56%  of  the  reports  that  were  traced  were  used,  this  mhy 
be  a  significant  and  positive  finding.  Other  technology  base 
areas  do  not  fare  any  better  with  the  utilization  of  their 
reports.  A  second  point  is  that  all  research  and  development 
does  not  find  its  way  into  eventual  utilization.  A  very  valuable 
payoff  from  research  and  development  can  be  in  the  form  of 
negative  findings,  which  stop  the  research  itself  or  stop  the 
development  of  some  system.  Third,  the  GAO  review  was  confined 
to  reports  published  from  1973  to  1975,  and  it  is  the  opinion  of 
the  present  authors  that  some  of  the  38%  not  used  will,  eventually 
find  some  utilization.  One  has  to  be  more  patient  when 
estalllshing  the  relationship  between  research  and  utilization. 

The  second  study  was. also  conducted  in  1975,  by  the  National 
Heart  and  Lung  Institute  (NHLI)  (see  Comroe  and  Dripps,  1975) , 
and  was  concerned  with  the  top  10  clinical  advemces  in  diagnosis, 
prevention,  and  treatment  of  diseases  of  heart,  blood  vessels, 
and  lungs.  This  was  a  four-year  study  to  analyze  what  knowledge 
was  required  for  the  great  advances  since  the  early  1940s.  Over 
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150  experts  screened  4,000  scientific  articles  and  then  analyzed 
529  of  these  that  they  considered  to  be  essential  for  the  top  10 
clinical  advances.  Of  the  529  key  articles: 

•  41%  reported  research  that  at  the  time  was  unrelated  to 
the  later  clinical  advance  (non-targeted  research) 

•  61.8%  described  basic  research 

•  21.2%  were  clinical  investigations  {targeted  research) 

•  14%  were  concerned  with  the  development  of  appiaratus 
techniques  or  procedures 

•  The  key  articles  range  in  time  over  200  years,  with  manv 
important  ones  being  published  as  long  as  75  years  ago. 

The  important  points  about  this  study  are  fairly  obvious.  At  the 
time  it  was  performed,  41%  of  the  reported  research  was  unrelated 
to  the  problem  it  later  helped  to  solve.  And  21.2%  of  the 
research,  while  clinical  in  nature,  was  unconcerned  with  the 
fundamental  issues.  Also  notable  is  the  range  of  time  (200  years) 
that  the  eventually  related  research  covered. 

The  NHLI  investigators  also  point  out  that  a  major  defect 
in  education  and  science  is  the  perpetuation  of  the  "one  man 
equals  one  discovery"  myth  (e.g.,  Marconi  equals  wireless; 

Bell  equals  telephone) . 

Inferences  about  the  technology  base  R&D  efforts  in  human 
factors  that,  are  based  on  the  NHLI  study  results  are  certainly 
limited.  Clearly,  the  quantitative  results  of  one  are  not 
applicable  to  the  other;  however,  the  difficulty  in  auditing 
or  tracking  cause-effect  relationships  is  similar. 


Illustrations  of  Contributions 
from  the  Technology  Buse 
to  Specific  System  Developu.ent 

To  conclude  the  discussion  of  human  factors  R&D  and  tne 
technology  base,  it  is  interesting  to  note  that  the  value  of  the 
research  conducted  in  each  cell  of  Exhibit  4-1  could  be  assessed 
by  the  contribution  that  the  research  makes  to  specific  system 
developments  over  the  years.  Again,  ho  matter  how  interesting 
it  may  be,  measuring  the  value  of  this  contribution  is  not  within 
the  scope  of  this  project.  Nevertheless,  it  is  meaningful  to 
qualitatively  emphasize  the  relationship  of  human  factors  in  the 
technology  base  to  system  development  products.  To  this  end,  we 
will  try  to  present  a  brief  illustration  for  each  cell  in  the 
matrix.  The  ilxustrations  which  follow  are  keyed  to  the  numbers 
in  the  cells. 

Block  1.,  Hximan  related  R&D  in  the  Mission  Analysis  Phase; 
measuring  human  tolerance  to  motion  to  assist  ship  design  and 
development. 

Need.  The  Navy  is  investigating  experimental  designs  of 
surface  effect  ships  (SES)  .  A  design  constraint  which  recjuires 
human  factors  R&D  previous  to  any  prototype  development  is 
determination  of  human  stress  tolerance  to  motion  and  associated 
human  performance  capabilities  while  in  moderate  and  high  sea 
states. 

Res  ear  ah.  Research  has  begun  on  human  tolerance  to  degrees 
of  SES  motion,  using  a  motion  generator  for  simulation.  In 
addition,  techniques  for  measuring  complex  human  performance  have 
been  developed  to  assess  the  ability  of  Navy  personnel  to  perform 
shipboard  tasks  during  extended  exposures  to  such  motion. 

Utilization.  This  human  factors  R&D  will  have  direct  impli¬ 
cations  for  the  design  of  SES  subsystems.  It  is  also , envisioned 
to  have  an  impact  upon  ship  opera tionS. 
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Block  2.  Human-machine  related  R&D  in  the  Mission  Analysis 
Phase  verifying  warning  system  audibility. 

Need.  In  the  absence  of  quantitative  data,  the  Air  Force 
was  concerned  with  the  adequacy  of  the  current  auditory  warning 
systems  to  offer  advanced  indication  of  a  missile  propulsion 
system  toxic  propellant  leak  to  operating  personnel  located  in 
missile  silos.  A  portable  vapor  detector  was  used  to  detect  such 
leaks  auid  to  sound  an  auditory  alarm.  Concern  was  expressed  over 
the  possibility  that  individuals  working  in  silos  with  ambient 
noise  levels  of  73  to  89  decibels  might  not  be  able  to  hear  the 
alarm.  Consideration  was  being  given  to  development  of  a  new, 
more  elciborate  warning  system. 

Reaearoh.  In  order  to  properly  evaluate  the  requirement  for 
a  new  warning  system,  human  factors  R&D  was  needed  to  determine 
the  adequacy  of  the  current  system,  before  initiating  development 
of  an  essentially  new,  alternative  system.  Toward  this  end,  a 
field  study  of  the  actual  system  was  performed  utilizing  oper¬ 
ational  personnel  to  report  when  they  heard  the  alarm.  Results 
indicated  that  the  personnel  heard  the  alarm  each  time  it  was 
sounded  cmd  made  no  false  reports. 

Utilization.  Consideration  of  possible  new  system  develop-r 
ment  was  abamdoned.  Cost  estimates  for  development  of  the 
alternative  system  for  500  silos  ranged  from  $250K  to  $1,000K. 

Cost  savings  achieved  through  elimination  of  em  unnecessary  system 
development  program  were  due  to  a  study  costing  aproximately 
$1,000. 
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Block  3.  Htunan-machine-mission  related  R&D  in  the  Mission 


Analysis  Phase:  Development  of  an  automated  command,  control, 

3 

communication,  and  intelligence  (C  I)  system. 

3 

Need.  In  order  to  develop  a  computer-based  C  I  system  for 
automated  battlefield  support,  the  Army  had  a  requirement  for 
hiiman  factors  R&D  to  develop  a  data  base  covering  mission  related 
human  performance  and  human-machine  concepts.  Due  to  unique 
problems  posed  by  such  a  system,  data  base  concepts  are  required 
which:  (a)  identify  human  user  input  and  output  capacities,  and 

(b)  offer  maximum  real  control  over  the  battle. 

Researad.  Simulations  for  automated  C^I  were  developed  to 
offer  expected  mission  scenario  rimulation,  identify  the  role  of 

3 

man  in  automated  C  T  systems,  demonstrate  operator  task  feasi¬ 
bility,  and  determine  optimal  design  criteria  for  the  man-machine 
interface.  Examples  of  results  include: 

•  Guidance  for  input  and  display  data 

•  Guidance  for  military  terms  abbreviation  to  reduce 
workload  eind  errors 

•  Use  of  embedded  training 

•  Comparisons  of  data  summary  methods  (e.g.,  graphic 
displays) 

•  Tradeoff  criteria  between  critical  data  retention  and 
expanded  data  retention 

•  Recommendations  for  data  reduction  and  purging  to 
facilitate  system  performance. 

Utilization.  Data  base  information  accrued  from  simulation 
research  on  the  automated  C^i  was  furnished  to  Army  developers  to 

aid  in  system  development  as  concept; guidance  and  criteria  for 

'3  .  '  V 

automated  C  I  systems.  Tnese  systems  are  expected  to  facilitate 
human  ability  to  govern  c.:>mbat  on  the  ground. 
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Block  4.  Human  related  R&D  in  the  Concept  Development 
Phase:  Visibility  requiremonts  for  underwater  information 
displays. 

Need.  Recent  advances  in  diving  technology  (e.g. ,  free- 
floating  manned  submersible)  have  created  a  requirement  for  human 
factors  R&D  on  underwater  vision  related  to  display  design. 

These  technology  advances,  tied  to  reduced  visibility  and  lack  of 
a  data  base  sufficient  to  guide  \inderwater  visual  display  design, 
combine  to  threaten  Navy  diver  mission  success  and  life  support. 

Reaeavoh,  The  research  progreim  that  was  initiated  encom¬ 
passes  both  display  requirements  and  basic  experiments  on  visual 
performamce  underwater.  Water  turbidity  simulation  techniques 
wer'”  developed  to  simulate  harbor  and  oceanic  waters.  Since 
number,  reading  and  signal  detection  were  identified  as  the 
most  critical  4isplay-oriented  tasks,  experiments  based  on  . 
these  parameters  were  constructed  that  revealed  the  following: 
brightness  was  the  strongest  legibility  factor,  green  was  more 
legible  than  red,  and  only  harbor  turbidity  had  a  major  effect 
on  legibility. 

Utilization.  This  continuing  line  of  reseaurch  has  resulted 
in  wide  distribution  of  information  to  both  reseeurch  and  fleet 
operational  communities,  whose  responsibility  it  will  be  to 
implement  these  findings  into  displays  to  be  used  in  ambient 
undersea  environments  such  as  instrumentation  in  nev  and  existing 
.diving  systems*  System- to-diver  and  diver- to-diver  communication 
will  also  be  facilitated  by  these  experimentally  derived 
guidelines. 
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duman- .machine  related  R&D  in  the  Concept  Devel¬ 
opment  Phase:  Selected  problems  in  armor  operations  and  design. 

A  numlier  of  problems  with  potential  impacts  on  arm.or 
operations  and  design  have  been  identified  Ly  the  .Army.  dxai.ples 
are:  concern  with  tlie  effects  of  external ,  environmental  conditions 

on  the  internal  environment  of  a  buttoned-up  tank,  and  concern 
with  the  adequacy  of  current  escape  and  evacuation  systems. 

.  In  order  to  assess  the  environmental  effects  upon 
tank  workspace,  data  on  internal  temperature  and  humidity  were 
obtained  using  a  recording  hygro- thermograph.  Chese  data  were 
compared  with  comparable  external  conditions.  Results  showed 
that  temperature  and  relative  hiumidity  inside  a'  tank  lag  behind 
the  external  conditions  by  approximately  throe  hours. 

In  addition,  opinion  data  were  obtained  from  crewmen 
concerning  the  adequacy  of  escape  and  evacuation  systems  and 
potential  design  changes.  One  conclusion  was  that  if  a  tank 
were  hit,  the  gunner  will  be  the  most  vulnerable  and  would  have 
the  greatest  difficulty  escaping.  Also,  revealed  was  that  lifting 
straps  snould  be  auded  to  uniforms  for  evacuation  of  wounded, 
and  that  escape/evacuation  training  was  extremely  limited. 

'.'til'lza  icn.  This  information  could  play  a  role  in  future 
design  oi  ne./  tanks  as  well  as  in  current  training  and  operations. 
Especially  in  t-'e  case  of  tank  crew  escape /evacuation,  design 
rocommendatiohs  could  have  an  impact. 


Block  6.  Human-nkachine-mission  related  R&D  in  the  Concept 
Development  Phase;  Effects  of  operator  interface  on  system 
cost-effectiveness. 

Heed.  The  lack  of  availadDility  of  a  reliable  technique  to 
aid  in  design  tradeoff  decisions  at  the  man-machine  interface  of 
system  development  has  resulted  in  a  Navy  need  for  computer 
models  to  simulate  operator  characteristics.  In  order  to  be 
useful,  such  a  model  must  be  eible  to  determine  whether  a  proposed 
or  potentially  modified  system  will  result  in  a  net  gain  in 
effectiveness  over  cost,  as  well  as  to  choose  the  most  cost- 
effective  alternative  means  of  achieving  a  specific  performance 
level,  when  the  operator's  job  is  considered. 

Reaearah.  Hiunan  factors  RSO  has  developed  an  Operator 
Interface  Cost  Effectiveness  Analysis  model  which  is  capable  of 
calculating  the  interactions  between  a  human  operator  (including 
control/display  location,  procedures,  decisionmaking,  observation, 
recall,  and  physical  movement),  system  hardware,  and  software, 
as  well  as  specific  mission  events.  It  has  been  applied  to 
evaluate  alternate  mission  equipment  configurations.  For  example, 
two  forward- locking  infrared  (FLIR)  sensor  configurations  for  the 
P-3C  surveillance  aircraft  were  evaluated  using  this  model. 

Results  showed  that  one  configuration  was  superior  due  to: 

•  Less  operator  disruption  in  other  tasks 

•  Less  time  to  perform  mission 

•  25%  less  costly  to  operate. ~ 

Utilination.  As  illustrated  in  the  example  provided,  this 
model  promises  substantially  improved  performance  for  Navy  m^m^ed 
systems  as  well  as  substantial  cost  savings  by  preventing' devel¬ 
opment  of  inferior  hardware  at  the  man-machine  interfaces.  It  is 
also  being  used  in  other  system  development  programs  for  the  Navy. 
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Block  7.  K’jman  related  R&D  in  the  Demonstration  and 
Validation  Phase:  Noise  limits  for  Army  materiel. 

"eed.  intense  sound  c»ccompanies  many  aspects  o,f  military 
operations  and  training.  Due  to  hearing  loss  which  occurs  as  a 
result  of  abusively  loud  noise  and/or  lack  of  effective  hearing 
protection  devices,  the  services  have  a  requirement  to  initiate 
neaiing  protection  and  auditoiry  research.  This  is  especially  so 
for  design- related  roseargh  with  intent  to  reduce  noise  through 
cierign  guidelines. 

Fesear^k.  The  Army  initiated  research  into  noise  effects, 
limits,  measurement,  and  testing  techniques  as  well  as  hearing 
prouect-on.  This  began  with  an  accumulation  of  existing  noise 
guid.: lines  and  other  information  (such  as  industry  standards 
covering  the  topic)  and  continues  today  with  research  to  fill 
gaps  in  hearing  technology.  Continuing  development  of  the  audi¬ 
tory  end  noise  data  base  has  resulted  in  the  initial  and  revised 
publication  of  MlL-STD-1474 ,  Noise  Limits  for  Army  Materiel. 
Plans  are  currently  underway  to  raise  the  standard  to  encompass 
DOD-widvi  application. 

i'til  ization.  The  use  of  this  standard  should  result  in 
a  significant  reduction  in  the  present  40-50  million  dollar 
ai.mK  1  expenditure  in  hearing  loss  compensation  paid  by  the  VA 
to  military  veterans.  In  addition,  this  noise  research  has  paid 
off  in  the  .dev-w-lopment  of  a  prototype  high  compliMce  idler  for 
tracked  vehicles  designed  to. reduce  noise  emanating  from  the 
axle  and  track  lodation. 


Block  8.  Human-machine  related  R&D  in  the  Demonstration  and 
Validation  Phase:  Human  factors  in  redesign  of  a  ground  infantry 
weapon  system. 

Need.  The  Army  had  a  requirement  for  human  factors  R&D  on 
the  DRAGON  antiteuik  missile  system.  Desired  we'>pon  improvements 
included  means  to  increase  target  "hit  rate,"  or  accuracy,  as  well 
as  to  make  it  more  portable.  Portability  was  important  given 
that  the  weapon  system  is  to  be  employed  by  ground  infantry  to 
counter  tank  threats. 

Research.  Hum2m  factors  R&D  was  implemented  to  investigate 
me2uis  to  improve  DRAGON  missile  system  accuracy.  These  efforts 
resulted  in  the  redesign  of  portions  of  the  weapon  system.'  For 
example,  a  lightweight  tripod/viscously  damped  mount  was  developed 
to  replace  the  previous  non-human  engineered  configuration.  In 
addition,  redesign  efforts  resulted  in  a  weapon  system  capable  of 
folding  into  a  lightweight,  compact  package  easily  portable  by 
one  individual. 

Utilization.  Field  tests  show  that  the  human  factors 
redesigned  DRAGON  weapon  system  yields  a  30%  increase  in  .hit  fate 
as  compared  to  the  previous  design.  The  tripod  modification 
provides  a  precision  tracking  capability  for  a  gunner  that  will 
substantially  improve  his  ability  to  hit  distant  moving  targets. 
Design  improvements  in  portability  zmd  compactness  will  facilitate 
transport  of  the  weapon  system. 
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Block  9.  Human-machine-inission  related  R&D  in  the  Demon¬ 
stration  and  Validation  Phase:  V/STOL  human  factors  planning. 

Need,  Navy  interest  in  developing  a  viable  V/STOL  aircraft 
nas  resulted  in  a  need  for  human  factors  R&D  to  support  human 
factors  design  of  the  aircraft.  One  reason  for  this  is  the 
taxing  workload  demanded  of  pilots,  creating  a  "pilot  factor" 
crucial  in  design  and  operation  of  this  type  of  aircraft. 
Contrary  to  expectation,  the  accident  rate  for  this  development 
program  was  also  increasing.  Pilot  factor  contributed  heavily 
to  this  problem. 

Research.  In  reaction  to  this  state  of  affairs,  the  Navy 
initiated  a  program  to  provide  human  factors  R&D  support  to  the 
human  factors  design  of  the  V/STOL  aircraft.  A  major  activity 
has  been  the  compilation  of  a  data  base  of  available  documen¬ 
tation  to  support  the  program.  The  data  base  identified  pilot 
workload  as  a  critical  issue.  A  "primer"  was  also  developed 
which  introduced  V/STOL  technology  and  operation  to  human 
factors  personnel. 

Utilization.  This  effort  has  contributed  to  the  human 
factors  data  base  for  V/STOL  aircraft.  In  addition,  human 
factors  personnel  involved  with  V/STOL  aircraft  design  were 
offered  job  performance  aids.  This  line  of  research  should 
contribute  to  an  ultimate  reduction  in  "pilot  factor"  accidents. 


Block  10.  Human  related  R&D  in  the  Full-Scale  Development 
Phase:  Information  display  for  landing  signal  officers  (LSOs) . 

Heed.  A  landing  Signal  Officer  (LSO)  standing  aboard  an 
aircraft  carrier  must  guide  a  carrier  pilot  and  aircraft  into  an 
appropriate  approach  to  a  landing  and  then  ma)<">  a  time-critical 
decision  (in  seconds)  as  to  the  safety  with  which  this  rapidly 
approaching  aircraft  m^iy  land  on  the  carrier.  These  decisions 
have-  traditionally  been  based  upon  a  limited  assortment  of  visual 
and  auditory  cues.  In  addition,  approach  and  landing  speeds  are 
high;  and  perceptual  cue  availability  is  adversely  affected  by 
night  and/or  adverse  weather  conditions.  Due  tc  annual  accident 
rates  associated  with  carrier  landings,  the  Navy  has  a  require¬ 
ment  to  aid  LSOs  by  developing  supplemental  information  displays. 

Reaearah.  Detailed  investigations  involving  task  analysis 
were  conducted  of  the  requisite  visual  and  auditory  cues  and 
associated  judgments  made  by  LSOs  when  guiding  the  aircraft's 
approach.  The  resulting  information  was  incorporated  into  an 
innovative,  see-through,  head-up  display  system  which  provided 
the  LSO  with  these  critical  parameters,  without  interrupting  his 
visual  tracking  of  the  approaching  aircraft. 

Utilizatior,  Operational  evaluations  were  conducted  on  the 
display  system  by. actual  LSOs.  Results  indicated  overwhelming 
approval  by  potential  users.  These  users  are  confident  that  the 
display  system  will  facilitate  safe  and  efficient  lamdin-j  oper¬ 
ations,  thus  reducing  accidents,  associated  losses  in  equipment 
and  personnel,  and  reduced  costs.  The  system  is  to  enter 
production  and  be  fielded  on  all  carriers. 


Block  11.  Human-machine  related  R&D  in  the  Full-Scale 


Development  Phase:  Man-machine  integration  technology. 

Need.  Due  to  advances  in  airframe,  flight  control,  and 
avionics  technologies  which  promise  to  revolutionize  aircraft 
capabilities,  the  Air  Force  has  a  requirement  for  human  factors 
R&D  to  improve  areas  of  man-machine  integration  technology  such 
as  aircrew  visibility.  Additional  areas  concentrate  around  a 
need  to  improve  tactical  aircraft  cockpit,  controls,  and  displays 
In  addition,  methods  to  measure  pilot  workload  under  different 
conf igurations  need  to  be  identified. 

Eeee  zvah.  The  feasibility  of  a  voice  activated  switch  of  a 
weapon  system  was  dtamonstrated  towards  improvement  of  aircraft 
cockpit  controls  and  displays.  These  results  led  to  a  request 
for  voice  controlled  switching  for  the  A-10  pilot  during  weapon, 
delivery.  In  addition,  an  improved  pilot/fire  control  interface 
combining  voice  activated  switching  and  helmet-mounted  sight  and 
fire  control  status  displays  to  facilitate  continuous  pilot  out- 
of-the-cockpit  vision  is  being  developed.  Methods  are  being 
developed  to  measure  pilot  workload  under  different  cockpit 
configurations.  Part  of  this  effort  has  produced  a  set  of 
symbols  that  present  order-of-battle  information  to  a  pilot 
rapidly  and  accurately.  . 

Utilization.  Improvement  to  aircraft  cockpit,  controls,  and 
displays  and  increased  data  base  development  of  pilot  workload 
capabilities  for  various  cockpit  configurations  has  immediate 
application  as  well  es  application  to  future  systems. 
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Block  12.  Human-machine-mission  related  R&D  in  the  Full- 
Scale  Development  Phase:  Humein  factors  R&D  support  for  FIREFINDER 
radar. 

Need.  The  Army  has  developed  the  FIREFINDER  radar  to  pin¬ 
point  enemy  indirect  fire  weapons.  Mission  sue  ess  is  contingent 
upon  a  fast  reaction  time  which  makes  human  performance  capability 
at  the  man-machine  interface  critical.  As  a  result,  himan  factors 
R&D  was  required  to  support  system  development. 

Research,  A  task  analysis  for  FIREFINDER  was  developed, 
tested,  and  refined  to  keep  up  to  date  with  system  hardware 
reconfigurations.  Army  human  factors  personnel  coordinated  with 
FIREFINDER  training  simulator  developers  to  support  design  of  a 
training  effective  simulation  system.  For  ejcample,  the  simulator' 
basic  training  effectiveness  was  confirmed  using  operational  per¬ 
sonnel  in  operational  test  and  evaluation  previous  to  delivery  of 
the  final  simulation  system.  Additional  deficiencies  were 
identified  and  corrected  at  the  contractor's  plant. 

Utilization.  The  FIREFINDER  task  analysis  is  in  use  by  the 
Army  for  developing  operator/maintainer  training  courses.  The 
simulator  now  in  use  by  the  Army  is  a  more  effective  trainer,  with 
the  pre-identified  deficiencies  corrected-,  than  it  might  otherwise 
have  been. 
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CHAPTER  5 


DERIVING  METRICS  FOR  MEASURING  THE  VALUE  OF 
HUMAN  FACTORS  IN  MILITARY  SYSTEMS  DEVELOPMENT 

The  previous  chapter  described  a  conceptual  basis  for 
identifying  the  contribution  of  human  factors  in  military 
systems  development.  The  major  question  to  be  addressed  now 
is:  Can  we  measure  that  contribution?  This  question  will.be 
answe.red  by  two  successive  discourses.  The  first,  presented 
in  this  chapter,  deals  with  metrics  for  describing  human  factors 
value.  The  second,  presented  in  the  chapter  to  follow,  develops 
a  methodology  for  measuring  human  factors  value. 

The  objective  of  this  chapter  is  to  demonstrate  that  system 
design  and  human  factors  criteria  and  terminology  are  compatible, 
and  that  a  vocabulary  for  human  factors  impact  assessment  can  be 
constructed  from  engineering  and  human  factors  by  means  of  common 
and  complementary  terms. 

This  chapter  presents  the  results  of  a  literature  review 
to  compile  terms  useful  for  describing  human  factors  R&D  products 
and  impacts,  compares  those  terms  with  conventional  system 
engineering  and  design  terminology,  and  derives  a  preliminary 
vocabulary  to  define  human  factors  R&D  impacts  on  military 
systems. 


I 


Background 

The  importance  of  objective,  quantitative  data  for  decision¬ 
making  in  the  system  develbpment  process  is  apparent  to  anyone 
involved.  The  presence  of  formal  mathematical  models  or  their 
rear  equivalent  has  permeated  every  level  of  system  development, 
from  the  engineering  draftsman  to  the  top  levels  of  the  Department 
of  Defense.  New  management  techniques  supported  by  quantitative 
measurement  have  evolved  rapidly  over  the  past  40  years. 

A  parallel  process  has  been  occurring  in  the  behavioral 
sciences  that  undergird  human  factors  applications.  There  has 
been  a  consistent  emphasis  during  the  past  40  years  on  rigorous 
measurement  and,  in  effect,  an  attempt  to  emulate  the  physical 
sciences  with  respect  to  precision. 

While  significant  strides  have  been  made  (for  example,  in 
scaling  techniques)  there  is  really  no  valid  prospect  that  the 
behavioral  sciences  will  ever  "catch  up"  to  the  physical  sciences 
in  the  matter  of  precision  because  of  the  inherent  character¬ 
istics,  such  as  high  variability,  in  the  phenomena  of  concern. 

This  circumstance  generates  a  chronic  problem  for  those 
concerned  with  the  contribution  of  human  factors  to  military 
systems  development,  in  current  parlance,  the  problem  is  the 
synthesis  of  "soft"  measures  from  the  psycho-physiological  domain 
of,  human  factors  with  the  "hard"  measures  presumably  available  to 
systems  engineers. 

This  study  establishes  a  basis  for  an  operational  synthesis 
of  those  measures.  The  three  essential  components  of  this  basis 
are: 
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1. 


A  forma]  relationship  of  explicit  human  factors  products 
with  the  system  development  process.  Chapter  3  develops 
the  relationship  of  principal  human  factors  products  for 
the  major  system  development  phases.  The  premise  of  the 
relationship  is  that  both  human  factors  and  systems 
engineering  are  components  of  the  system  development 
process. 

t 

2.  A  methodology  with  techniques  that  integrates  "soft"  and 
"hard"  measures.  The  methodology  presented  in  Chapter  6 
is  designed  for  the  formal  treatment  of  quantitative  and 

.  qualitative  impacts. 

3.  A  set  of  metrics  to  describe  human  factor  impacts  and  to 
define  parameters  for  the  modeling  process.  This  chapter 
addresses  the  derivation  of  metrics  for  describing  and 
measuring  human  factors  impacts  on  military  systems 
development. 
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Identifying  and  Defining  Metrics 


The  discussion  in  previous  chapters  and  in  DOD  documentation 
leads  to  the  assertion  that  any  human  factors  or  design  engineering 
change  must  ultimately  be  assessed  in  terms  of  its  contribution 
to  the  system-mission.  At  the  system-mission  level,  three  impact 
areas  have  been  identifiedr  cost,  capability,  and  compatibility. 
These  impact  areas  represent  categories  to  aggregate  or  embed  all 
the  effects  on  the  military  system  of  separate  design  and/or 
operations  support  analyses.  They  are  intended  to  represent  the 
"bottom-line"  effects  of  system  changes  (or  choices) ,  and  we  use 
them  in  this  derivation  of  metrics  as  the  most  broad  terms  in  a 
measurement  vocabulary  hierarchy. 

The  three  impact  areas  represent  distinctive  effects  of  a 
change  on  the  system-mission.  However,  the  effect  from  a  single 
human  factors  or  design  engineering  change  will  often  be  relatable 
to  several  or  all  of  the  Impact  areas.  Notionally,  an  improvement 
in  opera tor- system  compatibility  could  enhance  the  mission-system 
capability  and  also  lead  to  fewer  operator- induced  repair  actions, 
thereby  reducing  repair  costs. 


The  terms  at  the  loweist  level  of  our  hierarchy 
empirical  measures.  They  are  representative  of  th 
variables  that  human  factors  researchers  have  used 
but  such  empirical  measures  are  not  always . easily 
with  system  effects.  Therefore,  it  was  decided  th 
intermediate  level  in  our  measurement  hierarchy  thi 
to  both  system  developers  and ^ human  factors  profes 
intermediate  level  of  terms  is  defined  as  metrics 


illustrates  the  three  levels  of  our  hierarchial  mjeasurement 
Vocabulary. 


represent 
e  dependent 
for  many  years; 
identifiable 
at  we  need  an 
at  is  acceptable 
sionals.  The 
Exhibit  5rl 
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liicrarchit^al  Relationship  <»!' Impact  Areas. 
Metries,  and  Kmpirieal  Mea.siires 


SPECIFIC 


The  desirable  characteristics  of  metrics  have  been  determined 
to  be  the  following: 

1.  More  specific  than  impact  areas  (i.e.,  the  principal 
focus  or  objective  of  an  analysis) . 

2.  Representative  of  the  direct  result  of  an  analysis  or 
experiment. 

3.  Relevant  to  the  human  factors  principal  products. 

4.  '  Compatible  with  the  conventional  terminology  of  system 

engineering. 

5.  As  mutually  excluTive  as  possible. 


Char dcteristics  i  and  2  imply  that,  a  metric  is  the  output  of  a 
model  or  exper i:;it;nt  or  some  other  antiysis.  In  these  cases,  the 
iiietrio  is  a  function  of  some  lower  lev'el  parameters  or  empirical 
measures.  Characteristic  3  implies  th^- 1  metrics  must  be  relevant 
u.'.u  auitaoie  'co  Lite  auman  factors  erforts.  The  metrics  must  be 
r.:inuliar  to  n;r.tar.  factvi^rs  researchers  and  practitioners  and  allow 
for  realistic  interpretation  of  the  ci.anges  under  consideration. 
Cnaracteristic  4  implies  that,  measurements  comon  to  systems 
envj ineer ing  and  numan  factors  are  to  be  represented  by  a  single 
metric.  Thus,  system  engineering  and  human  factors  considerations 
are  integrated  within  common  metrics.  This  clearly  has  implica- 
ti-CiiS  for  the  modeling  of  these  integrating  metrics,  a  subject 
vijscussed  elsewhere  in  this  report.  Characteristic  5  is  a 
desirable  property  that/  if  satisfied,  implies  that  def initionaily 
one  metric  does  not  overlap  with  another.  We  recognize  that  this 
characteriscic  is  a  par (.icularly  difficult  one  to  satisfy,  and  is 
one  that  will  only  be  approximated  by  our  preliminary  vocabulary. 

These  characteristics  were  interpreted  as  selection  criteria 
in  the  derivation  of  the  preliminary  vocabulary. 

Literature  Search  and 
Dcta  Base  System 

We  used  an  empirical  approach  to  derive  the  metrics.  It 
entailed  collating  actual  terminologies  in  the  fields  of  human 
factors  research,  human  factors  engineering,  and  systems  research. 
The  literature  review  was  followed,  by  the  development  of  a  data 
base  tailored  specifically  to  human  factors  in  military  systems 
development.  Traditional  publications  for  human  factors  research 
provided  a  wealth  of  data  on  specific  empirical  measures  and 
metrics..  Computer  printouts  were  obtained  from  the  National 
Technical  Information  Service  (NTIS)  and  the  Defense  Technical 
Information  Center  (DT*C) .  Documents  containing  extensive 


*D i t) j, i o ^ jTi^pr* i c; s  woxc:  roir  pGirtiinsnt  oxtiQ^xons •  In 

addition,  an  intensive  effort  was  ,uade  to  identify'  and  obtain 
pertinent  government  directives,  instructions,  standards,  and 
guidance  docuiTienis.  This  literature  review  resulted  in  the 
identification  of  more  than  350  relevant  documents.  Subject 
matter  categ-  lies  i,iGluded:  human  factors  engineering;  costs; 
military  system  aevelopments/acqui sitions ;  test  and  evaluation; 
man-machi:.e  studies;  and  system  analysis,  design,  and  development 

The  products  of  the  literature  searches  were  used  to 
develop  a  uata  base.  The  data  base  was  the  primary  instrument 
used  to  analyze  the  selected  material  and  derive  a  set  of 
empirical  measures  and  metrics.  The  following  general  docu¬ 
mentation  caregcrios  were  accumulated  for  purposes  of  review 
and  assessment,  and  were  put  info  the  computer  data  base: 

1.  Technical  docu-mentation--including  technical  reports, 
papers,  memorandums,  bibliographies#  professional 
jour..al  articles,  ao'd  technical  books. 

2.  Policy  documentation--including  Department  of  Defense 
directives;  instructions;  pamphlets;  military 
specifications  and  standards;  and  individual  service 
instructions,  regulations,  and  pamphlets. 

2a.  Gu’ dance  documentation — including  DOD  and  indi¬ 
vidual  service  military  handbooks,  guidebooks, 
manuals,  and  pamphlets. 

3.  Work  Unit  Summaries  (DOD  Form  1498). 

4.  Informal  documentation — including  proceedings  of  various 
cum  ur ances  and  meetings,  unpublished  literature,  zmd 
personal  testimonies. 
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Screening  and  Selection  Process 

The  can  iidate  terras  derived  from  the  documents  collected 
were  sorted  and  grouped.  The  first  screen  applied  was  a  frequency 
of  use  check.  Ail  low  frequency- count  terms  were  reviewed  for 
usage;  terns  not  explicitly  described  by  the  author  were  deleted. 

The  terns  renaining  after  the  first  screening  were  then 
defined  using  the  following  procedure: 

1.  Quant irati'-’G  raeasures  were  defined  according  to  (a)  their 

'it  cf  measure  or  dimension  and  (b)  constraints  (e.g., 
t  ne  period,  events,  cycles,  etc.)  and  special  circum¬ 
stances  for  their  use  (if  any) ,  such  as  unique  usages 
(e.g. , ,  location) . 

2.  Qualitative  measures,  subjectively  determined,  were 
pvjt  through  additional  review.  Checks  were  also 
made  to  determine  if  such  measures  had  a  quantitative 
counterpart  or  could  be  tied  to  some  underlying  dimen¬ 
sion.  Whenever  characteristic  constraints  and  special 
circumstances  for  use  of  these  measures  were  discussed 
in  the  original  document,  this  information  was  included 
in  the  analysis. 

3.  Following  this  procedure,  the  list  was  ex^ined  to 
ensure  that  sufficient  iiiformation  was  available  about 

'  each  t€?rm  to  avoid  ambiguity. 

Derivation  of  Metrics 

The  next  step  entailed  the  derivation  of  metrics.  Criteria 
reflecting  the  five  metric  characteristics  were  applied  to  the 
list  of  empirical  measures.  A  partial  list  of  candidate  measures 
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and  associated  definitions  is  given  in  Exhibit  5-2  (the  complete 
list  is  in  Appendix  B) .  A  list  of  the  derived  metrics  is  given 
in  Exhibit  5-3.  In  each  list  measures  and  metrics  common  to 
system  engineering  and  human  factors  have  been  grouped  as  system 
related,  and  those  principally  used  in  human  factors  grouped  as 
personnel  related.  The  list  of  metrics  is  a  preliminary  one,  and 
in  subsequent  analyses  and  case  studies  it  should  be  refined. 

Each  of  the  metrics  is  relatable  to  several  empirical  measures. 
Exhibit  5-4  illustrates  several  of  the  functional  relationships 
observed  in  the  literature  reviewed.  ' 


Kxiiiiiii  5-2 

Sainpit'  of  Syslfm-Rflatcd  T«  rnis  and  Assooialcd  Dimensions  (I  nit  of  Mcasiirr) 


ACCESSIBILITY 


ACCURACY 

CAPABILITY 


COMPATIBILITY 


CRITICALITY 


DURABILITY 


EASE  OP  USE 


FAILURE  RATE /FREQUENCY 


tubiectwe:  Mtitfactory/unsatisfactory  ease  of  ad¬ 
mission  to  various  areas  of  a  s  item 


probability/frequency  of  documented  error 


subjeaivc:  mtuion  objective  achievable  given  the 
condition  during  the  mission 


subjective:  ability  of  items  of  equipment  to 
coexist  (including  effects  of  temperature  and 
moisture 


subjective:  rel.ttive  degree  of  tasJt  importance  fo- 
mission  succes 


probability:  item  nvill  survive 
al  its  projected  life 

b)  overnaul  point 

c)  rebuild  point 

without  a  dwability  failure  (failure  that  causes 
an  item  to  be  rebuilt  or  replaced 


subjective:  tasks  associated  with  simplicity,  reada¬ 
bility,  etc. 


1 1  number  of  failed  items 

2)  number  of  effects  (out  of  toleranoetl  per  month, 
^  week  hour  etc 


Exhibit  5-3 

Preliminary  Metrics  Derived  from  the  Literature 


SYSTEM  RELATED  METRICS 
AVAILABILITY 
BELIABILITV 
READINESS 
DEPENDABILITY 
EFFECTIVENESS 
MAINTAINABILITY 

PE RSONNE L  ACCOMMODATION/ENHANCEME  NT 
DESIGN/PRODUCTION  (PRODUCIBILITY) 
SYSTEM  RUGGEDNESS 
OPERABILITY 


PERSONNEL-RELATED  METRICS 
human  PERFORMANCE 
SKILL.  GENERAL 
SKILL,  maintenance 
TASK  LOADING 
PHYSIOLOGY/PERCEPTICN 
ENVIRONMENTAL  FACTORS 
OPERATIONS  FACTORS 

MOTIVATION/SOCIAL/ORGANIZATION  FACTORS 


Not*:  Thii  list  reprtstnts  »n  initial  atttmpt  t(f  construct  •  tat  o>  matrict  for 
human  factors  and  tys.am  angmaaring  All  the  tyttam-ralatad  matrict 
•re  common  to  •nginetring  end  hum«n  ftctori.  tnd  the  perionnfi' 
reieted  metrics  ere  p'ir>cip«Mv  refuted  to  human  lectori. 


Because  of  the  high  saliency  of  the  cost  issue,  the  inherently 
quantitative  properties  of  cost  assessment,  and  the  relatively 
low  ambiguity  associated  with  the  cost  concept,  the  derivation  of 
cost  metrics  and  measures  was  more  straightforward  than  f or ' the 
other  metrics. 


The  literature  of  system  evaluation  contains  many  detailed 
cost  structures.  It  was  convenient  to  use  the  generic  life  cycle 
cost  structure  developed  by  Fiorello  and  Betaque  (1977) .  That 
structure  reflects  current  usage  in  the  DSARC  process,  and  is 
presented  in  Exhibit  5-5. 

Each  of  the  impact  areas  is  discussed  in  the  next  section. 
Brief  illustrations  are  given  of  the  relationships  among  derived 
empirical  measures,  metrics,  and  impact  areas. 

Exhibit  5*5 

System  Life  Cycle  Cost  Impact  Area 
for  a  Weapon  System 

SYSTEM  LIFE  CYCLE  COSTS 

100  RESEARCH  AND  DEVELOPMENT 

200  INVESTMENT 

201  Weapon  System  Investment ' 

202  Support  Investment 

300  OPERATING  AND  SUPPORT 

301  Deployed  Unit  Operations 

302  Below  Depot  Maintenance 

303  Installations  Support 

304  Depot  Maintenance 

305  Depot  Supply 

306  Second  Destination  Transportation 

307  Personnel  Support  and  Training 

308  Sustaining  Investments 

Sourc*  Fi«rtllea  B«iagu«.  1977. 

.  atMi:  In  th«  tarmmolotv  uMd  in  ttiii  raport,  lift  eycl*  CMt  it  lti« 

.  imptei  •/«•;  int  ceti  cattaoritt  tt  titt  100,  300,  and  300 
Itvtlt  art  andogout  io  COM  mairict;  and  til*  lowar  lM«l  catt 
aiamantt  art  analoaout  to  ampirical  maawrM. 
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Impact  Areas,  Metrics,  and  Related  Measures 
Capability  Impacts 

The  impact  area  called  capability  has  a  clear-cut  tie  to  the 
man-machine-mission  performance  of  a  new  system. 

There  are  many  excunples  of  the  linkages  between  the  measured 
capability  of  the  human  and  the  ultimate  performance  of  the 
system  during  the  mission  operations.  Many  different  categories 
of  behavior  have  been  examined  in  many  different  system  contexts. 
A  particularly  good  illustration  can  be  derived  from  the  focus 
on  the  human  operator  in  a  controlling  or  decision-making  role  in 

3 

command,  control,  commuhications ,  and  intelligence  (C  I)  systems. 
Specifically,  the  case  involves  radar  signal  processing.  Radar 
system  functions  (i.e. ,  missions)  include  ground  control  of 
tactical  air  strike  operations,  interceptor  operations,  and 
return- to-base  operations  in  adverse  weather  or  poor  visibility 
conditions. 

So-called  "raw"  radar  returns  are  indiscriminate  in  the 
sense  that  the  indication  of  the  position  of  one  aircraft  on 
the  radar  display  looks  just  like  the  indication  of  every  other 
aircraft  in  the  same  coverage  load.  Under  low  traffic  loads, 
radar  controllers  are  able  to  "keep  track""  of  the  identity  of 
the  aircraft  represented  by  a  particular  return  "in  their  heads." 
Under  medium  loads  .(approximately  seven  simultaneous  "targets") 
or  higher,  the  controller  becomes  prone  to  errors  of  identifi¬ 
cation,  and  system  performance  consequently  degrades  rapidly. 

» 

During  the  early  applications  of  radar,  it  was  possible  to 
compensate  for  this  capacity  limitation  by  the  use  of  manual  plot 
boards  or  plotting  teddies.  The  job  of  directing,  the  movement  of 
interceptors  was  divided  into  three  main  requests.  The  radar 
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operator  (1)  reported  "target''  positions  to  a  plotter  (2)  who 
moved  coded  markers.  The  controller  (3)  made  decisions  based 
on  the  representational  display  provided  by  the  plotting  board. 

In  the  mid-1950s,  it  was  conceived  that  it  was  possible 
to  put  a  computer  between  the  radar  signal  and  the  operator. 

The  computer  could  be  given  the  burden  of  "remembering"  the 
identity  of  a  target  and  generating  a  display  such  that  the 
identity  was  reliably  associated  with  position..  There  followed 
a  major  sequence  of  human  factors  contributions  that:  estab¬ 
lished  precisely  what  the  consequences  were  for  overall  system 
effectiveness,  specified  how  much  information  should  be  tied  to 
each  target  (e.g.,  altitude  as  well  as  identity),  directed  the 
formatting  and  coding  of  the  information,  and  defined  backup 
procedures  in  the  event  of  a  subsystem  failure. 

Empirical  measurement  in  this  case  was  carried  out  primarily 
in  the  context  of  real-time  simulation  experiments.  The  specific 
variables  measured  were  the  average  delay  in  transit  during 
return- to-base  operations  and  error  counts,  defined  by  the 
instance  of  two  aircraft  coming  into  a  predetermined  proximity 
relationship. 

The  metric  level  in  this  case  would  be  represented  primarily 
by  effectiveness  and  operability.  These,  in  turn,  %fould  feed 
into  the  impact  area  of  capability. 

Cost  Impact's 

» 

At  the  mission-systems  level,  the  cost  impact  area  is 
defined  by  the  life  cycle  cost  of  the  system;  in  other  words, 
the  total  cumulative  cost  of  bringing  a  system  into  being  and 
using  it  over  its  operational  life. 
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Hunicin  factors  engineering  typically  accounts  for  a  small 
fraction  of  the  cumulative  cost.  The  outlay  of  funds  is  used 
to  support  the  generation  of  human  factors  products,  during 
the  design  stages  of  system  development,  and  for  specialist 
participation  in  test  and  evaluation. 

The  specific  engineering  inputs  from  human  factors  sources 
may  impact  the  development  and  production  costs  (both  positively 
and  negatively).  For  example,  it  is  possible  that  the  layout  of 
displays  and  the  controls  on  an  instrument  panel  that  is  optimum 
from  a  human  factors  point  of  view  can  be  more  or  less  difficult 
to  fabricate  during  production  or  could  require  more  or  less 
expensive  components.  However,  it  is  actually  more  likely  that 
what  turns  out  to  be  optimum  from  a  human  factors  viewpoint  is 
also  optimum  with  respect  to  economy  of  production.  Moreover, 
the  possible  additions  to  cost  tend  to  be  relatively  insignifi¬ 
cant  when  compared  to  some  of  the  cost  avoidcmce  potential 
inherent  in  successful  human  factors  contributio.,s . 

Any  number  of  actual  or  hypothetical  cases  could  be  cited  in 
which  human  factors  considerations  contributed  to  a  significant 
reduction  in  life  cycle  costs.  For  excunple,  the  size  of  the 

operational  crew  for  any  given  weapon  system  is  a  question  that _ 

links  hviman  factors  considerations  to  economic  consequences.  The 
larger  .the  crew,  the  higher  the  life-cycle  costs  will  be;  but  an 
arbitrarily  small  crew  might  not  be  able  to  handle  peak  work  . 
loads  during  crucial  mission  stages.  Work  load  capacity  limi¬ 
tations  are  the  kind  of  specific  products  that  can  be  generated 
in  a  rigorous  manner  through  the  proper  application  of  human 
factors  procedures.  Such  estimates  should  not  be  made  by  rule- 
.  of-thumb  or  guesswork. 


The  quality  of  performance  (inverse  of  error  frequency) 
on  the  part  of  crew  members  in  different  crew  size  and  organi¬ 
zational  arrangements  can  measured  empirically.  The  context 
of  measurement  can  vary  from  rough  task  simulation  in  laiboratory 
settings  to  observations  in  the  field  of  actual  operations. 

The  results  of  such  observations  can  contribute  to  the  overall 
system  design  deliberations  in  several  areas,  but  in  the  present 
framework  the  critical  linkage  is  to  cost  factor  301,  costs 
associated  with  deployed  unit  operations  (see  Exhibit  5-5) . 

Once  that  linkage  has  been  made,  the  logic  of  aggregation  to 
overall  life  cycle  cost  is,  straightforward. 

The  final  point  to  be  made  is  that  the  cost  impact  could  be 
forecast  with  little  ambiguity  once  the  crew  size  decision  has 
been  made. 

Compatibility  Impacts 

Compatibility  is,  in  general,  the  most  complicated  of  the 
three  impact  areas.  It  is  complicated  because  there  are  three 
distinct  but  interrelated  facets  involved;  physical,  physio¬ 
logical,  and  psychological  compatibility. 

Physical  Compatibility.  Physical  compatibility  relates  to 
the  human  component  as  a  physical  object.  The  major  resource  for 
the  conduct  of  tests  and  analyses  of  physical  compatibility  is 
anthropometric  measurements.  An  example  can  be  drawn  firom  the 
design  of  the  cockpit  ejection  subsystem  for  a  high-performance 
jet  interceptor  aircraft.  The  problems  involved  in  such  a 
design  aure  many.  The  routine  for  activation  must  be  simple 
because  the  pilot  is  under  severe  distracting  stress  when 
activation  is  required.  The  ejection  module  must  clear  the 
aircraft  in  such  a  way  th' t  there  is  no  impact  with  the  aircraft 
structure  and  must  stabilize  so  that  parachute  deployment  is  not 
impaired. 


» 
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As  was  indicated  in  Chapter  2,  cockpit  size  is  shrinking  as 
a  general  trend.  One  ejection  subsystem  design  recently  proposed 
met  all  the  complex  requirements  except  one.  The  trajectory  of 
the  module  was  such ' that  as  the  pilot  cleared  the  cockpit  the 
first  few  inches  of  the  toe  portion  of  the  flying  boot  came  in 
contact  with  the  edge  of  the  instrument  panel.  The  impulse  force 
was  sufficient  to  shear  off  the  boot  tip  and  the  pilot's  toes 
with  it. 

In  this  case,  as  in  others,  it  requires  no  elaborate  com- 
putational  model  to  conclude  that  a  better,  more  comprehensive 
application  of  anthropometric  data  would  improve  the  design  of 
this  subsystem.  The  consequences  for  system  performance  are 
similarly  clear. 

Physiological  Compatibility.  It  is  convenient  to  use  the 
discussion  of  physiological  compatibility  to  make  the  distinction 
between  capability  and  compatibility  a  bit  clearer.  This 
distinction  can  be  accomplished  by  considering  the  human  factors 
design  parameters  of  an  armored  personnel  carrier.  In  such  an 
instance,  it  is  useful  to  temporarily  separate  the  functions  of 
the  operator  from  those  of  the  rider  or  passenger. 

The  mission  of  the  system  is  to  deliver  the  passengers  in 

an  unimpaired  condition  to  somfe  locale  under  hostile  conditions. 

The  key  attribute  of  the  passengers  is  that  they  are  not  in 

control  during  the  journey.  In  effect,  they  are  pa£<sive  cargo. 

But  there  are  many  ways  in  which  the  design  of  the  system  can 

$ 

be  either  compatible  or  incompatible  with  their  physiological 
characteristics.  For  example,  when  traversing  hostile  territory 
in  a  combat  situation,  the  vehicle. will  be  "buttoned- up"  in  the 
sense  that  all  hatches  and  doors  will  be  closed.  Such  a  closed 
environment  creates  a  potential  limitation  of  effective  fresh  air 
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circulation.  The  vehicle  itself  is  a  potential  source  of  toxic 
gaseous  air  contaminants  (e.g.,  carbon  monoxide).  Hostile 
action  can  generate  other  toxic  airborne  chemicals.  In  suph  a 
case,  the  human  factors  engineering  role  would  be  to  inventory 
all  possible  sources  of  toxic  contamination.  The  analysis  would 
include  a  full  range  of  adverse  circumstances  that  could  arise 
from  the  terrain,  weather,  and  hostile  action. 

It  is  conceivable,  but  unlikely,  that  such  a  design  review 
would  indicate  a  zero  hazard  potential  in  the  "buttoned-up"  mode. 

If  the  hazard  potential  were  present,  then  certain  design  options 
such  as  compressor-powered  ventilation  systems  could  be  evaluated. 
That  is,  the  cost  and  complexity  of  the  ventilation  process  could 
be  weighed  against  the  estimated  probability  of  a  hazard  arising 
in  operations,  the  intensity  of  such  a  hazard,  and  the  consequences 
to  the  mission  if  the  hazard  were  not  eliminated. 

'  During ■ the  design  and  prototype  stages  of  system  development, 
the  empirical  level  measures  would  come  "from  the  book"  in  the 
sense  that  the  human  vulnerabilities  to  toxic  compounds  are  well 
known  and  documented.  Other  parametric  considerations  such  as 
the  likelihood  of  inflammation  of  hydraulic  fluid  due. to  mishap 
or  enemy  action  would  have  to  come  from  other  members  of  the 
design  team.  During  field  trials  of  prototype  models,  however, 
a  reasonable  empirical  test  could  involve  the  actual  sampling 
and  chemical  analysis  of  the  air  in  the  interior  of  the  vehicle 
during  combat  exercises. 

Finally,  an  analogous  approach  could  be  tedeen  to  a  list 
of  other  potential  physiological  hazards temperature  extremes, 
noise,  vibration,  and  acceleration  stresses.  While  such  a 
thorough  analytic  and  empirical  review  might  be  considered 
burdensome,  the  potential  negative  consequences  of  the  delivery 
of  exhausted,  disoriented,  or  incapacitated  troopis  into  a  fire 
zone  are  clearly  worth  the  trouble  to  avoid. 


Psychological  CoiP.patibility .  Psychological  compatibility 
has  two  component  parts:  behavioral  and  attitadinal.  The  ques¬ 
tion  of  behavioral  compatibility  can  be  illustrated  easily  by 
an  example  of  the  use  of  technical  documentation  by  maintenance 
technicians.  From  a  system  developmient  standpoint,  the  initial 
empirical  data  needed  would  be  in  the  form  of  a  distribution 
function  of  the  reading  skill  levels’  of  the  populatioii  of 
assignable  technicians. 

The  rather  obvious  human  factors  recommendation  would  be 
to  match  the  difficulty  level,  and  in  particular  the  vocabulary 
of  technical  documents,  to  the  relevant  skill  level  of  the  user 
population  (i.e.,  the  tenth  percentile  or  the  second  stanine 
level  and  above) .  During  early  system  development,  design 
criteria  could  be  met  by  the  imposition  of  a  "control"  vocabulary 
on  the  preparers  of  the  technical  doctiments.  During  later  stages 
(validation  and  verification) ,  empirical  tests  of  the  readability 
could  be  conducted  using  representative  technicians  from  the 
prospective  user  population. 

Increasing  the  compatibility  of  technical  documentation 
can  result  in  substantive  imprpvemeuts  in  weapon  system  cost 
and  availability.  An  excunple  is  the  experience  of  the  avionics 
maintenance  improvement  teaun  with  the  avionics  suite  on  the 
F-lllD.  Re-test-OKs  were  averaging  over  44%  and  Were  reduced 
by  15%  due  to  improving  the  technical  documentation.  . 

Attitudinal  compatibility  is  somewhat  harder  to  illustrate, 

t 

and  it  is  a  matter  of  some  dispute  whether  or  not  even  strong 
negative  attitudes  on  the  part  of  a  system  operator  can  be  a 
significant  problem  when  that  operator  is  under  military  disci¬ 
pline.  However,  in  the  recorded  history  of  the  interaction  of 
military  personnel  and  their  equipment  there  have  been  instances 


wherein  system  fierforinance  suffered  because  of  the  users'  negative 
attitudes.  A  modern  example  of  user  acceptance*  or  attitudinal 
compatibility,  can  be  drawn  from  an  article  that  appeared  in  the 
Armed  Forces  Journal  (April  1978).  Because  of  its  brevity,  an 
exact  excerpt  of  the  article  is  included  below. 

ARMY'S  PIERRE:  "FIX  BAYONETS!" 

DOESN'T  COME  ACROSS  ON  THE  COMPUTER 

It  is  easy,  in  tossing  around  the  acronyms  and 
technical  terms  associated  with  developments  in 
command,  control,  and  communications  to  lose  sight 
of  their  purpose:  to  help  human  beings  perform 
their  tasks.  Dr.  Percy  Pierre  of  the  Army  puts  it 
this  way: 

'We  seem  to  have  disguised  equipment  that 
performs  reasonably  understandable  functions 
behind  a  variety  of  unpronouncable  acronyms 
and  names  (like  digital  multiplexer)  that 
are  only  meaningful  to  those  familiar  with 
the  echnology.’ 

Pierre,  like  so  many  others  consulted  for  this 
issue,  cautions  that  voice  transmissions  will  still 
be  required  on  the  battlefield,  no  matter  how  far 
into  the  future  one  projects,  or  how  extensively 
commands  and  information  are  converted  into  digital 
forms.  He  told  Congress  recently,  for  instance, 
that  "Follow  Me"  or  "Fix  Bayonets"  are  commands 
fhat  do  not  convey  the  same  impact  when  received 
in  a  computer  printout. 

The  human's  desire  to  hear  another  human  voice  in  a 
time  of  crisis  held  back  deployment  of  an  advanced, 
digital  device  a  few  years  back.  According'  to  experts 
at  one  of  the  leading  electronics  companies,  their 
engineers  had  developed  a  tiny  two-way  radio  device 
that  could  be  used  at  rifle  company  and  platoon  level 
to  send  and  receive  messages  in  formatted  fozmi.  So,- 
if  a  leader  wanted  to  call  for  artillery  fire  mission, 
he  just  punched  in  a  code  and  the  device  sent  it. 

He  received  acknowledgement  with  a  beep  signal. 

The  device  worked  well,  but  it  failed  in  field  tests. 

The  troops  didn't  want  a  beep  when  they  called  for 
help — they  wanted  to  request  the  fire  mission  by 
voice  in  order  to  express  their  urgency,  and  they 
wanted  a  calm  human  voice  telling  them  that  the 
rounds  were  on  the  way,  not  a  beep.  They  distrusted 
the  beep,  since  it  could  have  been  caused  by  a 
malfunction  in  the  system. 
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The  principal  message  in  all  of  the  preceding  examples  has 
been  that  it  is  possible  to  proceed  both  upwards  and  downwards  on 
the  "ladder"  of  measurement  aggregation.  Each  type  of  military 
system  will  involve  different  specific  measures  and  different 
patterns  of  linkage  from  one  level  to  another.  However/  it  is 
demonstrable  that  the  linkage  can  be  made.  The  analysis  can  be 
focused  at  the  level  of  discourse  employed  by  systems  designers 
and  program  managers  to  assess  the  systems  upon  which  they  are 
working.  The  procedure  by  which  specific  evaluation  of  the  human 
factors  contribution  can  be  made  is  discussed  in  the  next  chapter. 
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CHAPTER  6 


METHODOLOGY  FOR  EVALUATING  HUMAN  FACTORS  IMPACTS 
IN  MILITARY  SYSTEMS  OE’/ELOPMCNT 

The  objectives  of  this  chapter  are  to  provide - 

o  A  systematic  f rcimework  or  methodology  that  can  be  used 
to  measure  the  value  of  human  factors  in  the  development 
of  military  jjyst^s 

•  A  classification  for  different  techniques  or  models  that 
can  be  used,  within  the  above  freunework,  to  estimate  or 
reflect  human  factors  related  impacts  on  military  systems. 

The  first  section  of  this  chapter,  "Human  Factors  Impact 
Assessment:  Related  Concepts,  Limitations,  and  Considerations," 
provides  a  brief  overview  of  the  underlying  principles  and  method¬ 
ologies  used  to  fomraulate  the  recommended  freunework.  We  utilize 
the  concepts  of  cost-benefit  analysis  and  poiicy/iropact  assessment 
in  our  framework  and  attempt  to  tailor  those  methodologies  to  the 
human  factors  setting.  The  second  section,  "Human  Factors  Impact 
Assessment:  Conceptual  Freunework,”  presents  the  set  of  steps 
that  comprise  the  recommended  methodology.  Each  of  the  steps  is 
described,  and. in  several  of  the  steps  a  notional,  human  factors 
related  problem  is. used  as  an  illustration  of  the  process.  The 
third  section,  "Techniques  for  Estimating  Impacts:  Classification 
and  Selection,”  discusses  the  step  in  the  methodology  that  selects 
and  adopts  the  techniques  for  estimating  the  associated  costs  and 
impacts  of  the  human  factors  related  recommendations  in  military 
system  development.  A  basic  •classification  of  technlqiueii  or  models 
is  presented  that  can  serve  as  a  basi..  for  identifying  their 
characteristics,  thus  facilitating  the  selection/development  of 
the  most  effective  model  form  for  the  particular  human  factors 
impact  under  consideration. 


Human  Factors  Impact  Assessment: 

Related  Concepts,  Limitations, 
and  Considerations 

Our  focus  is  on  the  assessment  of  human  factors  applications 
in  the  context  of  military  systems  development.  Throughout  this 
multi-phased  development,  there  are  opportunities  for  applying 
human  factors.  The  major  areas  of  emphasis  of  the  human  factors 
R&D  products  were  discussed  in  previous  chapters  and  listed  in 
Exhibit  4-1.  During  these  "windows"  of  opportunity,  one  or 
several  human  factors  related  actions  can  be  considered.  For 
each  of  the  candidate  actions,  or  for  a  set  of  them  in  which  the 
individual  actions, are  not  separable,  the  quantitative  and  quali¬ 
tative  impacts  must  be  estimated.  The  human  factors  quantitative 
and  qualitative  impacts  can  then  be  compared  to  the  estimated 
impacts  of  other  non-human  factor  actions  (e.g,,  reliability 
growth)  to  assist  the  decisionmeiker  in  allocating  resources. 

The  underlying  concepts  used  to  formulate  this  human  factors 
impact  assessment  methodology  are  cost-benefit  analysis  and 
system  impact  assessment,  and  their  conceptual  source,  systems 
analysis. 

Cost-benefit  analysis  is  ,a  foimi  of  systems  analysis.  It  is 
a  method  for  deriving  relevant  information  about  the  desirable  and 
undesirable  effects  of  projects  or  alternative  actions  under  con¬ 
sideration.  The  approach  is,  in  general,  analytical,*  it  entails 
specifying  objectives  end  alternative  solvitions  and  selecting  the 
preferred  alternative  based  upon  its  relative  cost-benefit  rating. 

♦Excellent  discussions  on  the  theory  of  cost-benefit  analysis 
can  be  found  in  Anderson  and  Settle  (1977),  Fisher  (1971) ,  Quade 
(1975),  Mischin  (1976),  and  Prest  and  Tuvey  (1965).  Quade  (1975) 
provides  a  very  readable  overview  of  the  evolution  of  systems 
analysis  techniques,  including  cost-effectiveness  analysis 
followed  by  cost-benefit  analysis  followed  by  impact  assessment 
analysis. 
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In  theory,  this  is  just  what  needs  to  be  done  with  respect  to 
evaluating  the  contribution  of  human  factors  to  system  development. 

A  basic  premise  of  classical  cost-benefit  theory  is  that  all 
costs  and  benefits  are  expressed  in  monetary  units.  In  many 
applications  this  requirement  does  not  present  difficulties. 
However,  in  the  human  factors  setting  the  process  of  interpreting 
all  the  metrics  and  their  impacts  into  dollars  would  often  have 
to  be  done  in  an  arbitrary  manner,  and  that  could  distort  the 
analysis.  Presently  there  is  no  foolproof  way  to  treat  intangible, 
distributed  impacts  in  a  strict  cost-benefit, monetary  forecast 
framework.  In  general,  a  strict  cost-benefit  approach  for  human 
factors  analysis  will  not  be  practicable.  What  appears  more 
feasible  and  useful  is  an  extension  of  the  cost-benefit  approach 
known  as  impact  assessment. 

For  these  and  other  reasons  (e.g.,  inability  to  isolate 
individual  impacts) ,  it  may  not  be  feasible  to  establish  an 
unambiguous  ordinal  ranking — let  alone  a  cardinal  ranking— of  the 
alternative  actions  in  strictly  dollar  terms..  What  is  needed  is 
to  extend  the  cost-benefit  monetary  metrics  by  presenting  the 
additional  relevant  mpacts  and  metrics,  such  as  user  acceptance, 
in  their  natural  (non-monetary)  dimensions.*  Techniques  such  as 
system  impact  assessment  do  exactly  that.  The  impact  assessment 
matrix  representation'  is  a  systematic  array  of  the  information, 
including  non-monetary  measures,  and  useful  comparisons  can  be  ' 
made.  A  simple  illustration  of  an  impact  assessment  matrix  is 
shown  in  Exhibit  6-1.  We  note  that  in  any  instance  in  which  the 
qualitative  metrics  are  non-discriminatory  and  the  quantitative 
metrics  can  be  expressed  in  dollars,  the  impact  assessment  method 
reduces,  to  the  classical  cost-benefit  framework. 


*A  good  introduction  to  eiystera  impact  assessment  is  provided  by 
Goeller  (1976). 
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On  the  other  hand,  where  the  quantitative  metrics  are 
non-discriminatory ,  dependence  on  the  qualitative  metrics  is 
necessary.  Where  this  is  the  case,  various  techniques  can  be 
used  to  facilitate  the  comparisons.  One  of  the  simplest  is  to 
use  a  color  scoreboard  apr-roach  in  which  the  relative  rankings 
of  the  candidate  actions  are  indicated  by  colors  for  each  quali-^ 
tative  metric  or  measure.*  Usually  four  colors  are  used  (green 
for  best,  blue  for  next  best,  red  for  worst,  and  yellow  for  all 
the  others) .  More  complex  techniques  assign  relative  weights 
reflecting  importance  to  each  qualitative  metric  or  measure, 
utilize  ordinal  scales  for  representing  the  relative  ranking 
of  the  actions,  normalize  the  ordinal  scales,  and  translate 
the  products  of  the  relative  weights  of  tlie  metrics  and  their 
normalized  rankings  into  a  quantitative  rating.  These  and  other 
relevant  techniques  that  can  be  used  to  estimate  the  value  of  the 
metrics  in  the  cells  of  the  matrix  in  Exhibit  6-1  are  discussed 
later  in  this  chapter. 

Methodological  and  Data  Limitations 

The  impact  assessment  approach  avoids  a  fundamental 
limitation  associated  with  a  strict  cost-benefit  approach. 
However,  there  are  still  a  number  of  important  limitations  and 
considerations  tnat  must  be  dealt  with  when  attempting  to  assess 
the  potential  impacts  that  could  result  from  a  human  factors 
related  application.  These  limitations  and  considerations  are 
discussed  next. 


♦The  use  of  color  acts  as  a  reminder  that  the  scale  is  ordinal 
at  best,  and  helps  prevent  the  natural  tendency  to  overlook  the 
limitations  of  the  data.  See  Goeller  (1976)  for  illustrations 
of  this  approach  for  transportation  system  decisions.  The  color 
scoreboard  can  easily  incorporate  the  quantitative  measures  as 
well. 
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'  li^olating  Human  Factors  Impacts.  It  is  v'cz'y  difficult — 
and  frequently  impossible — to  accurately  isolate  the  individual 
impdccs  iron,  aggregated  impacts  when  the  human  factor  impacts 
are  not  independent  of  one  anotiier,  or  when  the  individual  con¬ 
tributions  to  the  overall,  aggregated  impacts  are  not  uniquely 
measurable. 

in  such  instances,  aggregating  all  the  concurrent  hiunan 
factors  related  actions  is  called  for.  When  the  impacts  from 
the  aggregated  human  factor  actions  cannot  be  distinguished 
accurately  from  the  impacts  of  the  non-human  factors  actions, 
an  approximate  attribution  of  the  total  negative  and  positive 
impacts  on  the  military  system  to  the  contributing  actions  is 
necessary.  A  conceptual  basis  for  such  attributions  can  be 
found,  for  example,  in  Saaty  (1979)  and  Ostrofsky  (1977) . 

Ltllizing  Sophisticated  Tecliniques.  Many  of  the  models 
that  can  incorporate  intangiple  mpacts  are  complex,  difficult 
to  use,  ami  not  straightforward  to  understa.nd.  iv’hen  a  complex 
procedure  is  needed  to  assess  the  causal  relationship (s)  between 
an  action  and  an  impact,,  it  will  often  be  necessary  to  employ 
e;.a:yti.ral  specialists  to  apply  the  technique  and  interpret  the 
re^ijils.  The  resources  needed  to  do  the  analysis  are  part  of 
the  cosh-impact  assessment  decisions. 

Component  vs.  System  Impacts.  Often  the  focus  of  the  human 
factors  P.iD  activity  will  be  on  an  individual  procedure  or  compo¬ 
nent  and  not  an  entife  system.  VAien  the  procedure  or  component 
is  changed,  us  a  consequence  \jf  the  human  factors  related  actions, 
the  i.mpact  should  be  related  to  the  system's  mission  capability, 
co3t,  or  compatioility .  It  is  often  difficult  to  relate  the 
results  of  an  analysis  of  a  part  to  the  whole.  In  many  such 
instances,  an  opportunity  cost  argument  for  the  "freed"  resources 
oi  improved  capability  is  the  most  appropriate  explanation  of  the 
impact. 
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Tracking  Impacts  from  Phase  to  Phase.  The  conceptual 
process  envisaged  calls  for  the  consideration  and  assessment 
of  human  factors  impacts  throughout  the  development  phases  of 
a  military  system.  There  are  officially  four  development  phases 
(as  per  0MB  Circular  A-109)  including  milestone  3 — the  production 
decision.  Each  phase  represents  a  windovf  of  ooportunity  for 
human  factors  related  actions.  The  impact  assessment  framework 
is  intended  to  be  applied  to  the  candidate  actions  within  each 
phase.  In  keeping  with  the  baseline  concept  used  in  cost-benefit 
analysis,  the  projected  impacts  are  evaluated  re] ative  to  a 
specified  baseline.  When  a  design  or  procedure  is  changed, 
the  baseline  for  subsequent  impact  assessments  is  also  changed. 
Consequently,  the  baseline  will  be  continually  updated  as  changes 
are  introduced  over  the  system  development  phases.  Thus,  an 
impact  forecasted  in  one  phase  will  not  necessarily  be  additive 
with  impacts  forecasted  (claimed)  in  earlier  or  subsequent  phases 
Impacts  forecasted  should  be  presented  and  documented  relative  to 
the  baseline  for  the  phase  in  which  they  are  generated,  and  not 
casually  aggregated  across  ph.>.ses. 


Differentiating  R&D  Funding  Impacts.  In  general j  it  will 
not  be  apparent  how  to  relate  in  a  quantitative  and  precise 
manner  the  different  R&D  categories  used. to.  fund  human  factors 
analyses  to  differences  in  the  resulting  impacts  bn  the  system 
design.  To  the  extent  that  the  R&D  budget  categories  and  t:he 
type  of  R&D  activity  are  defined  and  applied  in  a  consistei^t 
manner,  then  a  degree  of  differentiation  will  be  feasible. 


System  vs.  Non-System  Specific  Impacts.  In  general, 
not  be  apparent  how  to  estimate  in  a  rigorous  way  the  impact 
h: — 'n  factors  research  beyond  a  specific  weapon  system  sett; 
that  is,  to  classes  of  equipment,  or  general  military  proc 
This  is  particularly  true  for  "basic"  research.  (Note:  TH 
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probien  could  be  an  artifact  of  the  budgeting  procedures  used  in 
DOD.  A  di.-tinction  between  human  factors  research  {which  is  non 
system  specific)  and  human  factors  engineering  (which  is  system 
speciiic)  might  i  eiaove  the  problem  altogether.) 

Data  Limi  cations .  Two  recent  5:tudies,  Butler  (1979)  and 
Orlansky  (1979)  have  observed  that  there  is  a  lack,  of  sufficient 
honoqcneou.s ,  longitudinal  data  to  properly  formulate,  measure, 
and  validate  the  analysis  of  impacts,  (whether  human  factors 
based  or  not)  on  military  personnel  perforrr^ance  and  training.' 
This  limitation  raises  the  issue  of  feasibility  for  any  approach 
that  requires  extensive  data  or  that  does  not  generate,  store, 
and  measure  the  .required  data  as  part  of  its  analytic  design.  , 

Risk  and  Uncertainty.  In  general,  the  treatment  of  risk 
and  unC'Ur t.iinty  ir.  models  that  assets  impacts  is  not  adequate., 
Procedures  do  exist  tc  quanti,fy  and  incorporate  risk  in  cost 
and  tonefit  projections ,  See,  for  example,  Fisher  (1973)  ,  Beers 
(1997),  Sobei  (1965),  Murphy  (1970),  and  Diene.mann  (1966). 

Manpower  Policy.  Analytic  techniques  tend  to  raask  the 
military  manpower  policy  effects  on  candidate  design  changes 
generated  by  R&D  results  or  design  variations.  In  general,  a 
simulation  model  is  required  to  incorporate  the  impact  changes 
and  manpower  policy  requirements  in  a  consistent  framework - 
Suc.h  movlcls'  are  often  not  applicable  until  the  later  stages  of 
the  3;,stem  development  process. 

Considerations  for  )?uman  Factors  Practitioners 

An  important  step  in  the  development  of  a  methodology  is 
to  determine  the  set  of  characteristics  its  potential  users 
desire  for  it.  We  have  identified  three  basic  user  requirements 
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that  must  be  accoirjticxiated :  compatibility  with  the  system  design 
process,  compatibility  with  the  human  factors  R&D  process,  and 
practicality  of  the  tools. 

System  Design/Development  Context.  To  be  effective,  the 
human  factor  impact  assessment  process  must  be  'rompatible  with 
the  system  design  and  development  process.  The  methodology  must 
facilitate  the  embedding  of  the  human  factors  contributions  and 
products  into  the  system  design  characteristics,  as  described  in 
Chapter  3,  and  into  the  systein  cost- performance  measures. 

A  number  of  operational  design  tools  used  in  cost-availability 
analyses  (see,  for  example,  Forster,  1974s'  Baran  &  Goclowski, 

1978;  Fabbro  &  Fiorello,  1977;  and  Fiorello  &  Betague,  1977) 
provide  frauneworks  to  relate  system  parameters,  such  as  reliability 
and  maintainability,  to  cost  and  performance  in  a  causal  manner. 
Those  tools  also  enable  the  design  changes  under  consideration  to 
be  ranked  in  terms  of  their  contribution  to  cost  reduction  and/or 
capability  enhancement.  Such  tools  provide  a  useful  design  . 
perspective  that  is  relevant  and  necessary  for  the  assessment  of 
human  factors  R&D  in  system  design. 

In  addition  to  the  design  context,  the  methodology  must  be 
compatible  with  the  changing  focus  and  characterization  of  the 
design  activity,  as  the  design  progresses  in  its  development  from 
the  conceptual  through  the  operational  stages.  In  the  earliest 
stages,  broad  macro  issvies  are  pertinent.  In  che  latter  stages, 
more  detailed  equipment  or  micro  issues  are  pertinent. 

Compatibility  with  Human  Factors  R&D  Issues.  To  be  useful 
in  the  human  factor  R&D  process,  the  methodology  must  be  relevant 
and  suitable  to  that  process.  It  must  utilize  familiar  parameters 
and  allow  for  realistic  interpretation  of  the  impacts.  Further, 


specific  tools  must  be  capable  of  handling  the  multi--objective 
and  multi-constraint  settings  typical  of  the. human  factor  and 
manpower  setting.  This  consideration,  when, fully  developed  into 
an  operational  screening  criterion,  will  be  useful  to  ideiitify 
and  select  compatible  models. 

Pragmatic  Procedures.  In  addition  to  being  compatible 
with  the  system  design/development  process  and  the  human  factor 
RiD  perspective,  the  methodology  must  also  be  practical  to  use. 
The  trend  in  contemporary  system  analysis  is  to  portray  evalu¬ 
ations,  in  as  rich  a  formulation  as  possible  (see  Quade,  1975; 
and  Goeller,  1976)  to  provide  the  decisionmaker  with  a  full  set 
of  relevant  information.  This  is  done  by  retaining  the  important 
natural,  multi-dimensional  impacts  and  presenting  them  in  the 
evaluation.  The  advantage  of  this  approach  is  that  changes  to 
the  system  cah  be  exam’.ned  not  only  in  monetary  terms  but  also 
in  terms  of  their  other  non-monetary  impacts.  The  major  drawback 
is  that  these  procedures  are  becoming  more  and  more  complex,  and 
they  tend  to  require  many  explicit  judgments  for  the  various 
multiple  impacts.  Ostrof sky's  (1977)  design  morphology  is 
certainly  a  case  in  point,  in  that  it  provides  a  systematic 
portrayal  of  pertinent  design  issues;  it  can  accommodate  human 
factor  parameters  (through  appropriate  surrogates) ,  but  only  at 
the  expense  of  extensive  detail  and  complex  computations. 

As  model  complexity  increases,  so  do  the  shill  requirements, 
the  difficulty  of  interpretation,  and  the  efforts  to  validate  and 
gain  acceptability  for  the  methods. 

Also,  a  pragmatic  methodology  should  be  flexible  enough  to 
use  on  a  breadth  of  human  factor  related  projects.  It  should  be 
readily  available  and  reasonably  convenient  to  apply,  have  low 


setup  costs,  and  not  require  unrealistic  data  inputs.  The  intent 
is  to  develop  these  requirements  into  discrete  criteria  for  use 
in  the  selection  and  application  of  candidate  models. 

Lastly,  the  methodology  should  help  structure  the  relationship 
between  the  human  factors  practitioner  and  decisionmakers — 
particularly  Program  Managers.  For  example,  anecdotal  reports 
suggest  that  there  have  been  incidents  in  Which  disagreements 
have • centered  on  such  issues  as  whether  a  design  decision  could 
be  made  on  the  basis  of  existing  principles  or  some  relatively 
informal  tests,  as  opposed  to  a  series  of  relatively  elaborate 
experiments.  A  good  methodology  would  be  one  which  helped  the 
parties  to  such  divergent  views  reach  an  agreement  without  rancor. 
In  other  words,  the  methodology  would  be  perceived  by  all  parties 
of  interest  as  providing  a  fair  test  of  the  time  and  resource 
investment  options  in  a  given  design  decision  situation. 

Rigor  vs.  Broad-Based  Analysis 

A  fundamental  issue,  underlying  many  of  the  above  points, 
is  whether  the  analysis  should  be  relevant,  rigorous,  and 
statistically  complete,  or  primarily  relevant  (descriptive  and 
broad) .  A  rigorous  evaluation  requires  (a)  formal  problem 
statements,  (b)  definition  of  the  analysis  and  testing  process 
within  a  communicable  model  framework,  (c)  the  capacity  for 
replication  by  different  analysts  at  different  times,  (d)  evalu¬ 
ation  designs  dependent  upon  the  use  or  availability  of  baseline 
or  control  groups,  and  (e)  that  the  number  of  observations  and 
the  number  of  model  relationships  are  both  greater  than  the 
number  of  test  characteristics  or  variables  of  interest. 

The  notion  of  broad,  relevant  studies  is  used  here  to  imply 
a  broad-based  analysis  where  the  intent  is  to  describe  what  has 
taken  place  or  is  expected, ■ to  identify  the  predominant  issues 
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in  a  certain  setting,  and  to  incorporate  them.  Many  relevant 
variables  cannot  be  measured  in  a  rigorous,  quantifiable  manner 
(for  example,  user  acceptance  and  variations  in  skill-mix) . 

This  dichotomy,  although  somewhat  contrived,  is  pertinent 
to  the  definition  of  the  cost-benefit  or  impact  analyses.  This 
is  so  because  not  all  human  factors  issues  or  parameters  carl  be 
analyzed  in  a  rigorous  manner.  This  limitation  on  rigorous 
analysis  must  be  dealt  with  explicitly  in  a  tradeoff  decision 
during  the  formulation  step  of  the  analysis. 

Dealing  with  the  above  limitations  in  itself  requires 
management  and  analysis  resources.  It  is  important  to  recognize 
what  the  relative  estimated  costs  are  of  the  evaluations  and 
the  impacts.  If  the  costs  of  the  analyses  are  comparable  to 
the  expected  value  of  the  impacts,  then  it  is  likely  that  the 
analysis  as  defined  is  inappropriate. 


Hiiman  Factors  Impact  Assessment: 

Conceptual  Framework 

Basic  Framework  and  Steps 

Exhibit  6-2  outlines  the  basic  impact  assessment  framework. 
The  development  and  presentation  of  the  analysis  entails  ten 
steps  or  phases.  The  steps  are  presented  in  a  logical  sequence, 
in  three  groups;  but  in  any  one  analysis,  as  indicated  by  the 
dotted  lines,  it  may  be  necessary  to  repeat  several  steps  in 
different  sequences  to  refine  perceptions  and  assessments  of 
critical  issues.  Each  step  will  be  discussed  in  some  detail 
below. 


1.  Establishing  the  Problem,'  Goals,  and  Criteria.  The 
objective  of  this  step  is  to  isolate  the  specific  issues  to 
be  analyzed,  to  bound  the  requirements,  to  specify  the  specific 
goals  and  objectives,  and  to  derive  the  decision  criteria.* 

Fisher  (1971),  Quade  (1975),  and  Goeller  (1976)  provide  useful, 
generic  guidance  for  this  step.  Specifically,  this  step  defines 
the  content  and  purpose  of  the  human  factors  product  to  be 
developed.  The  principal  human  factors  products  are  listed  in 
Exhibit  3-1,  earlier,  and  Exhibit  6-6  at  the  end  of  this  chapter. 

This  step  is  one  that  should  be  recognized  as  a  variation 
on  the  generic  system  analysis  method.  The  rule  is:  look  at 
the  ends  first  and  work  back  from  those  ends.  Ip  human  factors 
terminology,  we  would  probably  prefer  the  sequence:  Goals, 
Objectives,  and  Outcome  Measures  (or,  for  the  latter,  Dependent 
Variables) .  However,  the  principle  of  going  from  the  broad  to 
the  narrow  and  the  idea  of  a  hierarchy  that  includes  more 

*In  this  discussion,  the  term  goal  represents  an  "end,”  objective 
a  "means"  (that  is,  a  specific  accomplishment  within  an  explicit 
time  or  cost  target) ,  and  criteria  represent  specific  decision 
conditions. 
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Exhibit  6-2 

Impact  Assessment  Framework 
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"variables"  as  that  move  is  made  is  a  cono&on  linJc*  This 
arrangement  is  illustrated  in  Exhibit  6-3  in  a  particular 
cost-benefit  impact  assessment  convention  (Goeller,  1976; 
Ostrofsky,  1977) . 

The  following  hypothetical  example  illustrates  the  use  of 
goals,  objectives,  and  weights.  Assume  that  the  system  under 
consideration  is  proposed  for  the  XYZ  main  battle  tank.  The 
majof  goal  is  to  achieve  an  armored  fighting  unit  that  could 
defeat  its  hostile  counterpart  in  certain  tactical  scenarios. 

The  objective,  could  be  that  the  frontal  armor  would  hold 
against  80%  of  main  round  hits  .(i.e.,  any  grazing  angle  greater 
than  ±5°) .  The  objective,  O2,  could  be  to  achieve  an  average 
first-round  time  advantage  of  3  seconds.  In  this  case,  0^  could 
receive  an  a  priori  value  weight  somewhat  higher  than  0, . 

Criterion  €2^^  (contributing  to  objective  O2)  could  be  a 
maximum  turret  traverse  rate  of  =>20°  per  second.  Criterion  C22 
could  be  a  maximum  elevation/depression  rate  of  =>45°  a  second. 

In  this  case  the  criteria  might  be  assigned  equivalent  value 
weights. 

Several  attributes  of  the  hierarchical  setup  should  now 
be  clearer.  Specifically,  as  one  moves  down  the  structure, 
the  objective  measurability  improves.  But  more  importantly, 
the  actual  assumptions  about  perfotmance  are  made  very  explicit. 
That. is,  the  design  assumption  clearly  is  that  if  a  given 
elevation/depression  rate  and  a  given  traverse  rate  are  achieved, 
a  given  first  round  time  advantage  will  result.  Not  only  is  that 
assumption  measurable  (e.g.,  by  computer  simulation),  but  the 
tentative  weight  assignment  is  also  similarly  measurable. 

Computer  simulation  would  permit  a  whole  range  of  permutations  on 
the  traverse  rates  and  elevation/depression  rates  to  be  explored. 
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Exhibit  6-3 


Goal  Relevance  Tree  Hierarchy  of 
Goals  -  Objectives  -  Criteria 
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and  a  very  close  approximation  of  the  relative  importance  of 
one  to  the  other  could  be  obtained.  Moreover,  the  weights 
could  be  revised  as  other  kinds  of  testing  were  done. 

A  notable  gap  in  the  eibove  synthetic  scenario  is  the 
lack. of  explicit  consideration  of  the  human  factors  aspects 
of  sighting  and  firing  the  main  armament.  Fcr  example,  human 
factors  questions  would  arise  edsout  the  compatibility  of  a 
maxim’om  80°  traverse  rate  with  the  human  factors  requirement 
(hypothetical)  to  lock-on  to  a  target  on  the  first  traverse 
with  no  waver.  Human  factors  engineering  solutions  based  on 
traverse  deceleration  rate  damping,  sight  reticule  size,  etc., 
would  need  to  be  fitted  into  the  goal  and  ob jective-attainm.ent 
relationship  as  constraints .  The  basic  message  here  is  that 
it  might  not  pay  to  have  a  relatively  high  traverse  rate,  if  it 
led  to  an  overswing  of  the  turret  9  times  out  of  10  because  the 
rate/velocity  dynamics  were  incompatible  with  normal  human 
(psychomotor)  tracking  capabilities. 

The  characteristics  of  the  appropriate  set  of  goals, 
objectives,  and  criteria  is  critical  to  the  effectiveness  of 
the  analysis.  Several  useful  discussions  on  this  process  are 
provided  by  Fisher  (1971),  Quade  (1975),  and  Ostrofsky  (1977) i 
The  input-output  matrix  technique  used  by  Ostrofsky  (1977) 
appears  to  be  a  particularly  useful  way , to  structure  this  step. 
An  illustration  of  the  matrix  is  shown  in  Exhibit  6-4.  The  row 
headings  define  the  user  and  the, system  major  phases,  and  the 
column  headings  define  the  requirements  and  bounding  or  con¬ 
straining  conditions  (e.g. ,  resources).  The  row  headings  used 
in  Exhibit  4-1  could  also  be  used.  Ostrofsky  has  used  this 
format  to  formally  incorporate  Inanan  factors  considerations 
into  the  system  design  process. 
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Exhibit  6-4 

Input-Ciitput  Matrix  for  Problem  Formulation 
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The  output  from  this  step  is  a  problem  statement,  an  input- 
output  matrix  for  bounding  the  design-analysis  problem,  and  a  set 
of  weighted  objectives  and  decision, criteria  to  be  used.  The 
problem  state-ment  is  an  issue  that  one  or  more  huraan  factors 
related  actions  can  help  to  resolve. 

2.  Defining  the  Alternative  Solutions.  The  objective  of 
this  step  is  to  generate  a  set  of  explicit  strategies  or  alter¬ 
native  solutions  to  resolve  the  problem  or  issue  identified  in 
Step  1.  For  example,  within  the  human  factors  principal  R&D 
product — development  of  the  role  of  man  as  a  part  of  the 
mission — alternative  crew  sizes,  mission  flexibility,  and  system 
recoverability  could  be  specific  considerations.  There  are  two 
major  ways  this  can  be  done.  The  first  is  to  specify  a  set  of 
alternative  design  configurations  or  characteristics  or  process 
changes  at  the  subsystem,  component,  or  function  level.  The 
second  is  to  specify  a  criterion  function  (see  Ostrofsky,  1977) 
that  incorporates  the  design  parameters  in  a  mathematical 
function,  and  to  exercise  the  function  to  determine  the  preferred 
design  or  system  specification.  Either  approach  can  be  used. 

The  former  is  more  common  and  straightforward.  The  latter  is 
typically  more  rigorous  and  requires  more  definitive  analysis. 

Making  the  decision  options  explicit  is  a  fundamental 
principle  of  systems  analysis.  We  can  illustrate  this  principle 
In  the  context  of  using  cost-benefit  impact  analysis  to  measure 
the  impact  of  human  factors. 

Methodologies  such  as  cost-benefit  analysis  are  being  used 
increasingly  to  support  system  design  decisions  and,  to  a  lesser 
degree,  to  support  the  management  decisions  in  system  development. 
The  application  illustrated  here  includes  both  types,  but  empha¬ 
sizes  the  latter.  Management  decisions  of  special  interest  are 
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these  concerning  when  particular  inputs  to  the  design  deliber¬ 
ations  should  be  encouraged,  and  how  much  investment  to  make  in 
each  potential  source  of  such  inputs. 

For  illustrative  purposes,  than,  1st  us  say  that  the  range 
of  options  available  to  the  Project  Manager  with  respect  to  when 
to  encourage  human  factors  inputs  is  given  an  initial  framework 
by  the  four  design  phases  previously  defined,  i.e.: 

•  Mission  Analysis  (MA) 

•  Concept  Development  (CD) 

•  System  Demonstpatioh/Validation  (SD) 

•  Full-Scale  Development  (ED) . 

The  main  options,  then,  are; 

1.  None 

2.  MA  only 

3.  CD  only 

4.  SD  only 

5.  ED  only 

6.  MA  and  CD 

7.  MA  and  SD 


16.  MA  and  CD  ^u>d  SD  and  ED  (all). 

(In  the  higher-order  options,  the  question  of~  relative  degree  of 
input  becomes  a  factor--but  that  factor  overlaps  with  the  allo¬ 
cation  issue  and  adds  a  complication  that  is  not  needed  for  this 
illustration.)  Thus,  in  this  illustration  there  are  16  distinct 
^®^^>^*tives  for  when  human  factors  inputs  can  be  encouraaed. 

It  is  sufficient  for  this  step  simply  to  enumerate  them. 


3.  Specifying  the  Baseline.  The  objective  of  this  step 
is  to  define  the  status  quo  conditions  relevant  to  the  analysis; 
namely,  the  baseline.  Projected  impacts  are  evaluated  in  terms 
relative  to  a  baseline.  For  each  system  development  phase,  a 
systems  baseiino  must  be  defined.  Thus,  if  a  human  factors 
action  resulted  in  a  design  change  in  the  demohstration/validation 
phase,  the  baseline  for  the  succeeding  system  development  phase 
would  incorporate  that  change  because  it  had  already  been  accom¬ 
plished.  Thus,  the  baseline  is  generally  tied  to  a  phase  in  the 
development  cycle. 

The  baseline  provides  a  basis  for  the  projection  of  future 
conditions  in  which  the  human  factor  changes  under  consideration 
are  not  developed  and  implemented.  A  baseline  could  be  defined 
for  a  set  of  human  factor  impacts  when  the  individual  impacts 
cannot  be  isolated.  However,  it  must  always  be  defined  so  that 
the  impact  areas  and  metrics  under  consideration  are  explicitly 
identified. 

The  easiest  way  to  understand  this  step  is  to  make  the 
argument:  each  new  system  has  a  (more  or  less  direct)  precursor 
system  (or  systems).  The  baseline  rests  on. the  precursor  or 
composite  family  of  precursors  which  we  can  call  the  reference 
system.  In  most  instances,  the  reference  system  will  be  the  one 
that  would  be  used  to  perform  the  mission  if  the  new  system  were 
not  developed.  For  those  analyses  in  which  human  factors  are 
emphasized,  the  mission  compatibility  criterion  has  a  strong 
old-new  functional  similarity  aspect. 

The  following  discussion  illustrates  the  notion  of  the 
mission/functional  analysis  in  defini.ng  the  baseline.  There 
are  two  analytic  substeps  in  establishing  the  baseline  for  system 
design  and  cost  projection  pgrposes:  Functional  Differences  and 


Functional  Deficiencies.  The  first  entails  the  specification 
of  the  reference  system  similar  functions  and  any  technological 
differences  between  the  reference  system  and  the  proposed  new 
system.  For  example,  for  the  XYZ  tank,  the  reference  system 
would  be  the  operational  MYY  tank,  and  the  technology  differences 
that  impact  on  the  man-machine  functions  could  include  those  in 
the  main  armament,  armor  metallurgy,  turret  stabilization,  fire 
control,  and  propulsion  components.  The  functions  of  interest 
are  those  needed  to  operate  and  maintain  these  components.  The 
product  from  this  substep  is  a  reasonably  detailed  functional 
differentiation. 

The  second  substep  is  a  deficiency  analysis  of  the  reference 
system.  Again,  it  is  functional  deficiencies  that  count.  For 
example,  was/is  the  reference  system  deficient  in  maneuverability? 
In  what  specific  ways?  We  also  need  to  know  what  specific  human 
factors  related  deficiencies  were  brought  to  light  during  the 
field  use  of  the  reference  system.  Possible  source  data  for  this 
kind  of  deficiency  identification  could  include  the  complaints  of 
operators  and  maintenance  personnel.  Observations  of  the  actual 
behavior  of  crews  and  maintenance  units  in  action  could  also  be 
appropriate.  Also,  the  human  factors  specialist  could  actually 
go  through  dry  runs  of  crucial  segments  of  operational  and/or  ' 
maintenance  sequences.  The  product  from  this  substep  is  a 
definitive  list  of  deficiencies.  If  value  weights  could  be 
assigned  to  each  deficiency  in  a  unambiguous  manner,  this  could 
also  be  useful. 

The  baseline  is  completed  as  a  step  in  the.  overall  method¬ 
ology  when  the  array  of  technological  changes  and  reference 
system  deficiencies  are  put  together  in  such  a  way  as  to  give  a 
preliminary  picture  of  the  prospect  of  whether  the , technological 
changes  will  tend  to  ameliorate  or  accentuate  the  deficiencies  oh 
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a  one-by-ona  basis.  Thus  from  the  baseline  we  can  get  a  set  of 
assumptions  that  indicates  what  some  of  the  major  design  problems 
are  going  to  be  for  the  new  system  and,  importantly,  which  are 
likely  to  be  human  factors  related. 

4.  Preparing  the  System  Definition  Statement.  The  objec¬ 
tive  of  the  system  definition  statement  is  to  summarize  concisely 
all  the  essential  information  and  assumptions  about  the  subject 
system  that  are  necessary  to  conduct  the  impact  assessment. 

An  important  part  of  this  definition  is  a  historical  record  of 
tr.e  evolution  of  the  system's  design  and  development,  and  the 
corresponding  impact  and  cost  estimates.  Though  it  will  not  be 
possible  in  many  instances  to  aggregate  cost-benefit/impacts  from 
system  development  stage  to  stage,  the  definition  statement  can 
provide  selective  evidence  of  the  role  and  contribution  of  human 
factors  R&D. 

At  a  minimum,  the  system  definition  statement  should  contain 
specifics  on  the  following: 

•  Mission  Profile  (What  is  the  system  for?) 

•  ,  System  Performance  and  Operational  Characteristics  (V7hat 

are  .  the  sy.*'tem  capabilities?) 

•  Acquisitioh  Program  Schedule  (How  is  system  to  be 
procured?) 

•  Deployment  (Peacetime  and  Wartime)  Plan  (How  will  the 
system  be  utilized?) 

•  Support  Concept  (Initial  and  Mature)  (How  will  the  system 
be  supported  and  maintained?) 

•  Logistics  Goals  (What  are  the  unique  logistics  related 
goals,  e.g.,  reliability?) 
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•  Integrated  Logistics  and  Training  Considerations  (How  will 
the  operators  and  maintenance  personnel  be  trained?  How 
will  the  requirod  material  be  purchased,  managed,  etc.?) 

•  Human  Factors  Related  Issues  iWhat  operation  and  mainte¬ 
nance  considerations  can  affect  the  cost,  capability,  and 
compatibility  of  the  design?) 

The  first  seven  items  are  typically  called  for  under  current, 
recommended  major  weapon  system  acquisition  analysis  guidelines.* 
For  these  analyses,  we  have  augir.ented  those  guidelines  by  adding 
a  separate  disc  ission  of  human  factors  related  issues  that  should 
be  considered.  These  are  issues  that  would  be  noted  and  discussed 
in  the  human  factors  products  (e.g.,  role  of  man)  at  the  different 
system  development  phases.  The  outcome  of  that  consideration 
and/or  impact  assessment  should  be  reviewed  throughout  the  system 
development  stages. 


5.  Selecting  the  Impact  Areas,  Metrics,  and  Empirical 
Measures.  The  objective  of  this  step  is  to  define  the  system's 
life  cycle  cost,  capability,  and  compatibility  impacts,  metrics, 
and  empirical  measures  for  the  goals  and  criteria  identified  in 
Step  1.  Some  criteria  may  be  included  explicitly  as  cost  or 
empirical  metrics,  depending  on  their  specificity,  measurability, 
and  abstract  properties. 

Metrics  and  measures  used  to  define  the  specific  nature 
and  focus  of  the  h\aman  factors  R&D  impact  must  be  tailored  to 
the  phase  of  system  development,  the  human  factors  product  form, 

♦See,  for  exeunple,  DOD  Directive  5000.1,  Major  System  Acquisition; 
5000.39,  Acquisition  and  Management  of  Integrated  Logistics 
Support  for  Systems  and  Equipment?  and  DOD  Instruction  5000.2, 
Major  System  Acquisition  Process. 
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and  the  system-mission  characterization.  The  impact  area(s)  and 
associated  component  metrics  and  empirical  measures  comprise  the 
vocabulary  to  describe  the  effecc  of  the  human  factors  related 
change (s). 

The  three  generic  impact  areas — cost,  capability,  and 
compatibility — have  been  introduced  in  previous  chapters. 

In  Chapter  5,  each  of  the  impact  areas  was  shown  to  be  definable 
in  terns  of  a  number  of  metrics,  and  the  metrics  were  shown  to 
be  functions  of  combinations  of  empirical  measures.  The  generic 
hierarchical  relationship  was  illustrated  in  Exhibit  5-1. 
Moreover,  '_he  measures  and  metrics  for  capability  and  compati¬ 
bility*  ir  particular,  reflect  contemporary  usage  for  describing 
cause-effect  relationships  in  both  human  factors  R&D  and  system 
engineering.  In  general,  a  human  factors  related  change  that 
affects  capability  or  compatibility  will  also  affect  cost.' 

The  set  of  vocabulary  terms  presented  in  Chapter  5  are  from 
our  preliminary  findings.  They  represent  an  initial  step  toward 
the  definition  of  a  formal  and  stable  set  of  terms  to  discuss, 
model,  and  communicate  the  effects  of  human  factors  related 
changes  in  military  systems  design  and  development.  Each  of 
the  impact  areas  and  their  component  metrics  and  measures  are 
discussed  briefly  below. 

•  Cost:  For  a  weapon  system  specific  setting,  the  cost 
impact  area  is  the  life  cycle  cost  of  the  system.  A 
candidate  set  of  cost  metrics  (major  categories  of  costs 
such  as  Operations  and  Support)  and  measures  (cost  ele¬ 
ments  such  as  Below  Depot  Maintenance)  were  presented  in 
Exhibit  5—5.  If  a  military  system,  other  than  a  weapon 
such  as  a  C^I  system,  was  the  subject  of  the  analysis, 
it  is  likely  that  some  different  cost  measpros  would  be 


required.  The  quiding  criterion  is:  select  the  set  of 
cost  metrics  and  measures  that  reflects  the  significant, 
relevant  costs  effected  by  the  human  factors  related 
changes. 


9  Capability:  For  a  weapon  system  specific  setting,  the 

capability  impact  area  is  the  mission  worth  of  the  system. 
A  preliminary,  empirically  derived  set  of  capability 
metrics  (e.g.,  availability,  reliability)  and  measures 
(e.g.,  mean-time-to-repair,  mean-time-to-failure)  were 
presented  in  Exhibit  5-3.  The  particular  combination  of 
measures  used  to  functionally  define  a  metric  is  dependent 
upon  the  system  or  process  being  analyzed,  and  the  various 
ways  the  effect  of  the  hvunan  factors  changes  can  be 
measured. 


•  Compatibility:  For  a  weapon  system  specific  setting, 
the  compatibility  impact  area  is  the  physiological  and 
psychological  suitability  of  the  design.  A  background 
discussion  of  compatibility  metrics  (e.g.,  user  accep¬ 
tance,  motivation)  and  measures  (e.g.,  temperature, 
noise,  vibration  stress,  altitude)  is  given  in  Chapter  5. 
The  underlying  notion  of  the  compatibility  impact  area 
is  that  many  human  factor  related  effects  are  not  easily 
assessed  using  the  same  quantitative  metrics  and  measures 
as  for  cost  or  capability..  For  example,  reducing  an 
operator's  stress  is  a  substantive  benefit,  even  though 
its  contribution  to  enhanced  system  performance  is  not 
directly  quantifiable.  . 


The  result  of  this  step  is  a  specific  set  of  vocabulary 
terms  to  be  used  for  describing  the  impacts,  and  in  selecting/ 
constructing  a  model  to  estimate  the  values  for  the  measures, 
metrics,  and  ultimately  their  effects  on  the  impact  areas. 
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6,  Selecting/Constructing  the  Impact  Assessment  Models. 

The  objective  of  this  step  is  to  derive  or  select  appropriate 
techniques  or  models  that  can  provide  both  quantitative  and 
qualitative  measures  of  the  cost,  capability,  and  compatibility 
impacts  expected  from  the  application  of  the  human  factors 
change. 

In  effect,  one  needs  to  relate  the  criteria  from  Step  1, 
the  information  from  Steps  2  to  4,  and  the  impacts  and  Metrics 
from  Step  5.  Furthermore,  that  relationship  must  be  relevant 
to  human  factors  R&D  products  and  the  system  development  process: 
These  relationships  are  tailored  to  and  essentially  define  the 
content  of  the  human  factors  efforts  and  cells  in  Exhibits  3-5 
and  4-1,  respectively. 

A  reasonable  approach  is  to  utilize  Ostrofsky's  (1977) 
design  methodology  as  a  basic  procedxire,  and  to  augment  it  with 
other  models  that  deal  explicitly  with  life  cycle  cost  and  system 
capability  measures.  (Examples  of  the  latter  are  Goclowski, 

1978;  Forster,  1974;  Fabbro  &  Piorello,  1977;  AF-Logistics 
Support  Cost  .Model,  Design- to-Cost  Model,  and  the  Mission  Success 
Completion  Probability  Model.)  In  additioh,  there  are  several 
techniques,  other  than  Ostrofsky's,  for  evaluating  and  quanti¬ 
fying  (imposing  cardinal  measures)  on  essentially  qualitative, 
ordinal  measures.  Examples  are  Gardiner  (1979),  Saaty  (1979), 
Quade  (1975),  Hays  (1975) ,  Dalky  (1969),  and  Linstone  (1975) . 

Briefly,  the  sequence  envisaged  is  as  follows  (Os trof sky, 
1977);  . 
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a. 


For  the  criteria  defined  in  Step  1,  specify  the 
underlying  parameters.  These’ parameters  represent 
the  constituents  of  the  criteria  in  a  systems-component 
sense.  Each  parameter  is  classified  in  terms  of  being: 

-  measured  directly 

-  measured  from  a  mddel 

-  included  in  other  elements 

-  not  measurable  within  existing  resources. 

.  b.  Define  submodels  of  the  primitive,  measurable  elements 
to  define  functionally  the  higher-level  parameters. 

c.  Combine  the  submodels  into  an  overall  model  to  estimate 
each  criterion,  and,  in  turn,  an  aggregate  criteria 
function  for  the  overall  goal. 

While  each  of  these  steps  is  critical,  it  is  most  important 
to  understand  the  causal  linkage  between  the  elements,  which  can 
be  a  mixture  of  qualitative  and  quantitative  measures,  the 
parameter  submodels,  and,  in  turn,  the  criterion  function. 

For  a  "hard"  parameter  such  as  reliability,  the  linkage  between 
it  and  cost  and  availability  is  rather  well  understood >  and  many 
acceptable  models^ ^ist.  For  the  "soft"  parameters  such  as  user 
acceptance,  the  linkage  is  not  nearly  so  Clear.  What  is  required 
is  a  procedure  that  will  handle  both  quantitative  and  qualitative 
criteria  (and  their  parameters  and  elements)  in  a  syst6imatic  and 
credible  manner. 

In  summary.  Step  6  puts  all  the  information  from  Steps  1 
to  5  into  a  formal  setting  with  functional,  causal  relationships. 
From  the  previous  steps,  we  have: 


(Step  1) 

•  A  set  of  goals,  objectives,  and  criteria  in  a  hierarchical 
array. 

(Step  2) 

•  A  listing  of  (management)  decision  options. 

(Step  3) 

•  A  specification  of  the  baseline  in  the  form  of  an  explicit 
comparison  between  the  reference  system  and  the  proposed 
new  system  with  respect  to  technological  differences  and 
functional  deficiencies  in  the  reference  system,  and  pro¬ 
jected  implications  of  such  deficiencies. 

(Step  4) 

•  An  overall  characterization  of  the  proposed  new  system 
and  how  it  is  to  be  operated  and  maintained. 

(Step  5) 

•  A  listing  of  critical  metrics  and  empirical  measures. 

The  model  used  to  put  these  elements  together  can  take  a 
number  of  different  forms,  depending  upon  the  system  development 
phase  and  problem  setting.  A  discussion  of  model  types  and 
selection  criteria  is  given  in  the  last  section  of  this  chapter. 

We  can  now  proceed  to  summarize  the  final  four  steps. 

7.  Collecting  and  Processing  the  Data.  Given  the  specifi¬ 
cation  of  the  impact  areas,  metrics,  and  the  model  form,  this 
step  provides  the  required  data  to  "drive"  the  model.  Frequently, 
tJ-;:  lack  of  data  in  sufficient  quantity  or  detail  will  constrain 
the  nature  and  accuracy  of  the  cost-benefit  analysis. 


8.  Setting  the  Conventions  for  the  Analysis;.  This  step 
specifies  the  conventions  or  ground  rules  used  in  arriving  at 
the  cost,  capability,  and  compatibility  impact  estimatss. 
Conventions  for  cost  a;.d  capability  analysis  should  cover; 

a.  Normative  projections 

b.  Constant  versus  adjusted  dollar  cost  estimates/ 
projections 

c^  Mature  versus  transient  system  characteristics 

d.  Personnel  budget  or  economic  costs 

e.  Capital  investment  leadtime  considerations 

f.  Relevant,  variable  versus  total  costs 

g.  Uncertainty  analysis  (including  technical  risk) 

h.  Presentation  and  docvamentation  standards. 

9.  Estimating  and  Evaluating  the  Cost  Benefits.  This  step 
provides  the  output  from  the  model  and  data  prepared  in  Steps  7 
and  8 . 


10.  Presenting  and  Interpreting  the  Results.  This  step 
entails  preparing  the  presentation  (including  illustrations  and 
documentation  of  the  results) ,  identifying  the  requirements  for 
additional  analysis,  and  specifying  important  issues  that  have 
high  degrees  of  uncertainty.  An  important  part  of  the  presen¬ 
tation  is  a  description  and  quantitative  portrayal  of  how  the 
change  impacted  the  system  design  and  its  life  cycle  costs  and 
performance.  Where  feasible,  the  specific  contribution  of  the 
human  factors  change  should  be  isolated.  Often  it  m'.y  hot  be 
possible  to  Isolate  the  impact.  In  those  instances,  it  may  only 
be  reasonable  to  make  the  comparisons  at  the  aggregate  or  systems 
level  (e.g.,  new  vs.  baseline),  and  to- infer  the  role  of  the 
human  factors  impact.  In  addition  to  the  standard  tabular  and  , 


graphic  presentation,  the  notion  of  color  scoreboards,  as  used 
by  Goeller  (1976)  can  be  used  to  make  and  present  comparisons  of 
alternatives. 


Techniques  for  Estimating  Impacts: 

Classification  and  Selection 

Step  6  of  the  impact  assessment  methodology  is,  in  some 
ways,  the  most  crucial  and  the  most  complicated  of  the  10  steps. 
It  involves  the  selection  of  a  model  (or  several  models)  for 
conducting  the  particular  analysis  relative  to  the  attributes  of 
a  given  system,  the  human  factors  issues,  and  the  phase  of  system 
development. 

The  selection  process  is  complicated  by  the  fact  that  there 
is  no  one  model  or  model  type  that  is  appropriate  or  best  for 
analyzing  all  the  human  factors  issues  throughout  the  system 
development  phases.  Consequently,  we  camnot  recommend  any  one 
model  for  this  step.  Rather,  we  will  discuss  different  model 
types  and  scwve  suitable  selection  criteria. 

Because  there  are  a  number  of  models  that  can  be  used  in 
the  above  framework,  it  is  useful  to  have,  at  least,  a  way  to 
classify  model  types  in  terms  of  their  basic  characteristics. 

Our  classification  is  a  modification  of  the  one  used  by  Quade 
(1975).  The  model ' classes  are  not  mutually  exclusive;  that  is, 
a  model  assigned  to  one  class  can  also  have  seme  of  the  characr- 
teristics  of  models  in  other  classes.  The  class  name  indicates 
the  basic  or  principal  characteristics  of — and  the  means  used 
by~“the  model(s)  to  analyze  the  issues  under  consideration. 

We  have  identified  seven  model  classes  to  catalog  the  types  of 
models  that  are  potentially  useful  to  human  factors  analysis. 

1.  Mathematical  Models 

2.,  Computer  Simulation  Models 

3.  Experiments 

4.  Operational  Games 

5.  Surveys/Group  Decisions 

6.  Verbal  Models 

7.  Physical  Models. 
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A  sample  of  some  of  the  techniques  that  fall  within  the 
above  classes  is  provided  in  Exhibit  6-5.  As  this  methodology 
is  developed  and  appilied,  additions  and  deletions  to  the  sample 
list  will  be  made.  Of  the  classes  of  models,  these  used  in 
category  1  (mathematical) ,  category  2  (computer  simulation) , 
category  3  (experimental) ,  and  category  7  (physical)  appear  to 
be  those  used  most  frequently  in  human,  factors  analysis. 

A  suggested  set  of  selection  criteria  that  can  be  used 
to  identify  the  most  suitable  model  to  use  in  a  given  setting 
includes: 

1.  Validity  (Does  the  model  reproduce  or  realistically 
represent  the  functional  relationships  under 
consideration?) 


2.  Relevance  (Does  the  model  deal  explicitly  with  the 
human  factors  issues  under  analysis?) 

3.  Cost  (Is  the  model  very  expensive  to  construct  or  use?) 


Non-Trivial  (Does  the  model  prov'ide  substantive 


insights  into  the  process  under  analysis?) 

Feasibility  (Can  the  model  be  used?  Are  t 
required  available?  Are  the  staff  with  th 
s]cill8  available?  Is  there  sufficient  tim 
the  model?) 

Reliability  (Does  the  model  give  consisten 
under  different  circumstances?) 

Acceptability  (Can  the  model  results  be  co 
successfully  to  the  system  development  des 
managers?  Put  another  way,  can  or  will  th 
use  the  model  results?) 
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Exhibit  6-5 


Candidate  Models  and  Techniques  for  Human  Factors  R&D 
Cost  Benefit/Impact  Assessments  (sample  only) 


Model  Category 

Techniques 

1.  Mathematical  Models 

The  following  is  just  a 
small  sample  of  mathematical 
model  references: 

See  Goeller  (1976),  Hays, 

C Connor,  and  Peterson 
(1975),  Gardiner  (1979), 

Quade  (1975),  Fisher  (1971), 

Mood  (1974) ,  Petruchell 
(1963) ;  Hays  emd  WirXler 
(1970),  Draper  and  Smith 
(1966),  Baran  and  GoclowsJti 
(1978),  Fabbro,  Fiorello, 
emd  Shaw  (1977),  OstrofsJcy 
(1977),  Saaty  (1979),  BeUcer 
and  Pound  (1964) ,  Cetron, 

Martino,  and  Roepc)ce  (1967)  , 
Albnosta  and  Holzman  ( 1970) , 

Soudar  (1972),  Fisher  (1973), 
Martin  ^md  Sharp  (1973) , 

Beers  (1957),  Oienemann 
(1966)  . 

Systems  TUialyses  Techniques 

-  Systems  Impact  Assessment 

-  Policy  Analysis 

-  Sensitivity  Analysis 

-  A  Fortiori  TUialysis 

Statistical  (data  interpretation) 

Techniques 

-  Correlations  (cross-section) 

-  Regressions  (sioq>le,  multiple) 

-  Factor  Analysis 

-  Time-Series  Analysis 

-  Pooled  Cross-Section  Time-Series 

-  Parametric  Inferences  and 

Projections 

-  Multivariate  Analysis 

Military  Operations  Research  Models 

-  Logistics  Support  Cost  Model 
(AF-LSC) 

-  RelifUaility,  Maintainability, 
and  Availability  Tradeoff 

Models 

-  Material  Availability  and 

Resource  Investment  Models 

Types  of  Design  Aids 

-  Coordinated  Human  Resources 
Technology  (CHRT)  Model 

-  Life  Cycle  Cost  Impact  Model 
(LCCIM) 

-  Design  Morphology  (08trofs)cy, 

1977) 
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Exhibit  6-5  (Continued) 


Model  Category 


Techniques 


I.  Mathematical  Models 
(Continued) 


Types  of  Design  Aids  (Continued) 

-  Pair-Wise  Compeirisons 

-  Maintenance,  Reliability, 
Diagnostic  Accuracy, 

-  Availability,  and  Support 
Cost  Models 


2.  Coeiputer  Simulation 
Models 


Types  of  Cost  Models 

-  Plemning  Factor  Models 

-  Detailed  Engineering  Estimates 

,  Decision,  Rislc,  and  Utility  Theory 

-  Decision  Theory 

-  Risk/Uncertainty  Analysis 

-  Project  Scoring 

-  Utility  Scales 

-  Relevance  Tree  Techniques 
(Reverse  Factor  Analysis) 


Simulation-Process/Event  Flow 

-  Monte-Carlo  Techniques 
(DCOM,  CASES,  etc.) 

Moc)c-up8  (analogy) 


i .  Experimental  Methods 
See  Davies,  1967 


Types  of  Experiments 

-  One-Shot  Case  Study  (weakest  but 
most  coBBsonly  used  evaluation 
design) 

-  One-Group,  Pre-Test/Post-Test 
Design  (before  vs.  after) 

-  The  Static  Group  Comparison 
(with  vs.  without) 

-• Pre-Test/Post-Test ,  Control 
Group  Design  (before/after 
and  with/Without) 
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Model  Category 

Techniques 

3 .  Fjqperimer.tal 

Methods  (Continued) 

,  •  ■ 

Types  of  Experiments  (Continued) 

-  Solomon  Four-Group  Design 
(controls  both  the  experimental 
effect  and  the  possible  inter¬ 
action  effects  of  the  measuring 
process  itself) 

-  Post -Test  Only,  Control  Group 
Design 

-  Comparison  of  Alternative 

Program  Strategies  (with 
random  assignment' 

-  Non -Equivalent  Coinparison  Group 

-  Comparison  of  Alternative 

Program  Strategies 

-  Time  Series  Design 

-  Multiple  Time  Series  Design 

4.  Operational  Gaouis 

‘taming  Techniques 

-  War  Geunes 

-  Zero- Sum  Games 

Nominal  Group  Techniques 

-  Highly  Structured 

-  Homcgeneous  (e.g.,  only  planners) 

-  Small  Groups  (approx.  8-lQ) 

-  Basic  Steps t 

-  silent  generation 

-  round  robi;;  presentation 

-  clarification 

-  voting/ranking 

-  discussion  of  results 

-  Interpretive  Structural  Modeling 

-  9ex«onal  Interviews 

'  t 


5 .  Surveys/Groi^  Decision 
Models 

Sons  examples  are: 

Dailey  (1969)  linstone 
and  Turoff  (1975),  Morris 
(1977) ,  Brown,  Cochran; 
and  Dal)cy  (1969)  . 


Exhibit  6-5  (Continued) 


'  .  "I 

Model  Category 

■ 

Techniques 

5.  Surveys/Group  Decision 

Models  (Continued) 

Nominal  Group  Techniques 
(Continued) 

-  The  Delphi  Technique 

-  structured 

-  hierarchical 

-  quasi-emonymity 

6.  Verbal  Models 

Scenario  Building/Specification 
Analogy  Arguments  Dialectics 

7.  Physical  Models 

Seme  examples  are : 

.  Moder  emd  Hagers  (1968) , 

Miller  (1963) ,  Conway 
(1967) . 

Types  of  Scheduling  Aids 

-  Program  Eveduaticxi  and  Review 
Techniqi»  (PERT) 

-  Critical  Path  Models  (CPM) 

-  Functional  Flow  Diagrams 

-  Graphical  Evaluation  Review 
Techiniques  (GERT) 

-  Queuirtg  Models 

-  Programming  Techiniques  (linear, 
nonlinear) 

Moc)c-up3  (physical) 

As  we  gain  experience  applying  the  above  criteria  and  using 
the  selected  models,  a  more  specific  set  of  candidate  techniques 
can  be  provided.  Conceptually,  mb  need  to  identify  certain 
models  for  each  of  the  cells  in  Exhibit  6-6  and  indicate  their 
applicability  in  terms  of  the  selection  criteria.  That  informa¬ 
tion  could  be  made  available  in  a  handbook  that  would  be  updated 
on  a  regular  basis. 


The  scope  for  this  study  effort  includes:  the  development 
of  a  conceptual  basis  that  formally  relates  huiaan  factors 
efforts  and  products  to  the  major  phases  of  a  military  system 
development,  the  derivation  of  a  set  of  metrics  that  can  be 
used  to  specify  human  factors  impacts,  and  the  formulation  of  a 
methodology  to  quantify  the  impact (s)  of  human  factors  products 
on  the  system  design.  This  chapter  accomplishes  the  third  com¬ 
ponent  (Chapter  5,  the  second,  and  Chapter  3,  the  first).  It, 
presents  a  methodology  that  prescribes  a  set  of  practical  steps 
tailored  to  the  processes  of  human  factors  assessment.  Much 
remains  to  be  done,  hov/ever.  To  fully  develop  and  understand 
the  impact  quantification  process,  case  studies  are  needed  to 
demonstrate  tlie  methodology  and,  especially,  to  select  and  use 
pertinent  models  (Step  6).  As  experience  is  gained,  the  method¬ 
ology  can  be  refined  and  ttie  actual  results  categorized  for  use 
as  future  references.  A  useful  set  of  references  would  contain: 
(a)  a  refined  methodology,  (b)  case  examples,  and  (c),  data  and 
models. 
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Exhibit  6-6 

ModtrLs  and  Research  Techniques  Useful  in  Human  Factors  Analysis 


/ 
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CHAPTER  7 

CONCLUSIONS  AND  RECOMMENDATIONS 

This  study  sets  out  to  fexplore  and  define  a  conceptual  basis 
and  methodology  for  identifying  and  measuring  the  contributions 
of  human  factors  to  military  system  development. 

Our  major  conclusions  are; 

•  A  conceptual  basis  for  relating  human  factors  contri¬ 
butions  to  system  development  can  be  defined.  In 
Chapter  4  we  identify  and  relate  the  placement  and 
content  of  specific  human  factors  efforts  and  prcklucts 
with  the  development  phases  of  a  military  system.  Each 
phase  represents  a  window  of  opportunity  for  certain 
human  factors  products. 

•  The  human  factors  contribution  is  measurable.  In 
Chapter  5  we  derive  a  preliminary  (but  useful)  set 

of  metrics  for  describing  and  measuring  human  factors 
impacts  on  military  systems.  Actually,  a  three-level 
hierarchical  vocabulary  is  presented:  systems-mission 
terms  are  at  the  top,  for  cost,  capability,  and 
compatibility  impacts;  metrics  for  defining  the  primary 
,  focus  or  results  of  design  engineering  and  human  factors 
analyses  are  at  Level  2;  and  empirical  measures  are. at 
the  third  and  lowest  level.  We  also  show  that  system 
design  and  human  factors  criteria  and  terminology  are 
compatible  by  constructing  a  vocabulary  from  engineering 
and  human  factors  common  and  complementary  terms. 


•  A  methodology  for  evaluating  the  impacts  and  metrics  is 
feasible.  In  Chapter  6  we  build  upon  the  basic  concepts 
of  cost-benefit  analysis  with  recent  impact  assessment  . 
advances.  The  resulting  conceptual  framework  can  be  used 
to  evaluate  both  quantitative  and  qualitative  metrics 
needed  to  represent  human  factors  effects. 

Several  additional  steps  are  required  to  validate  these 
conclusions.  The  main  step  is  to  actually  exercise  the  proposed 
methodology  on  some  real-world  cases. 

In  order  to  support  such  test  applications,  it  would  be  most 
advantageous  to  first,  create  two  special  data  bases  in  the  form 
of  computerized  relational  files.  One  of  these  files  should  be 
an  expanded  and  slightly  restructured  version  of  the  file  used  in 
the  present  effort  to  establish  the  metrics  vocabulary.  The  new 
version  of  this  prototype  file  would  support  the  elaboration  and 
selection  of  the  specific  models  (techniques)  to  be  used  in 
operational  data  collection  and  analysis.  The  other  file  would 
consist  of  an  inventory  of  episodes  in  prior  and  ongoing  miilitary, 
systems  development  progreuns  (or,  ideally,  complete  chronologies 
of  work  programs).  The  objective  would  be  to  provide  the  means 
to  pinpoint  targets  for  the  impact  assessment  process. 

The  experience  gained  and  results  from  the  case  study  appli¬ 
cations  of  the  methodology  should  be  documented  in  a  series  of 
Human  Factors  Impacts  on  System  Development  Handbooks,  such  as: 

•  ..  Guidelines  for  analysis  that  would  present: 

-  the  conceptual  basis  for  relating  principal  human 
factors  products  and  military  system  development 
phases 

the  impact  assessment  methodology 
,  -  the  impact  assessment  vocabulary . 
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•  Models  and  techniques  that  would  catalog  selected  models 
used  to  assess  human  factors  related  changes. 

•  Reference  information  that  would  present: 

-  a  list  of  the  controlling  documentation 

-  cost  and  planning  factors 

-  description  of  the  relational  data  files  (discussed 
above) . 

•  Issues  and  case  studies  that  would  document: 

-  current  and  emerging  human  factors/system  design 
problems 

-  the  case  studies. 
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APPENDIX  A 


A  RATIONAI.E  AND  PRECEDENT  ,FOI?  ESTABLISHING  THE 
PRINCIPAL  HUMAN  FACTORS  PRODUCTS 
DURING  SYSTEM  DEVELOPMENT 


The  notion  of  principal  human  factors  products  resulting 
from  each  phase  of  system  development  is  a  cornerstone  of  the 
methodology  developed  in  this  report.  The  products  of  each 
phase  were  identified  in  Chapter  1  as  follows: 


MAJOR  PHASE 
OF  SYSTEM  ACQUISITION 


PRINCIPAL  HF  R&O  PRODUCT 


Mission  Analysis  Phase 


-  Development  of  the  Role  of  Man  as  a  part  of  a 
Mission  Element  Needs  Statement  (MENS) 


Concept  Developntent  Phase 


—  Allocation  of  System  Functions  to  Man  ta  a 
part  of  the  Decision  Coordinating  Paper  (DCP) 


Demonstretion/Validation 

Phase 


—  Task  Analysis  and  Determinetion  of  Huntan 
Enginaering  Raquiremants 


Fuil-Scaie  Davelopment  Phasa 


—  Design  of  the  Optimal  Man-Machine  Interfaces 


In  Chapter  3,  the  principal  human  factors  products  were  explicitly 
derived  in  terms  of  specific  human  factors  efforts  in  each  syst.em 
development  phase.  A  recommendation  about  the  types  of  infor¬ 
mation  included  in  documenting  each  product  was  also  provided. 

The  purpose  of  this  Appendix  is  to  provide  the  underlying 
rationale  and  precedent  for  establishing  the  principal  human 
factors  products  in  this  project.  Each  product  will  be  discussed 
separately. 
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Development  of  the  Role  of  Man 


Many  approaches  have  been  put  forth  for  analyzing  man's 
capeu^ilities  and  limitations  with  respect  to  his  potential 
role  in  system  performaince.  Some  approaches  suggest  that 
man  and  machines  should  be  compared  for  system  ,perf oinn2mce , 
while  others  suggest  that  man  and  machines  are  not  comparable 
but  are  complementary.,  Some  suggest  that  man  should  be  designed 
into  the  system  wherever  possible;  others  suggest  that  man 
should  be  designed  out  of  the ’ system  wherever  possible.  There 
are  numerous  controversial  issues  concerning  man’s  capeibilities 
and  limitations  for  system  performance.  The  philosophy 
expressed  here  (adapted  from  Price  &  Tcdsachnick,  1968)  is 
(1)  that  man  has  certain  unique  performance  capadsilities  that 
cannot  be  compared  with  machines;  (2)  that  many  system  perfor¬ 
mance  requirements  can  be  obtained  either  by  man,  or  man-machine 
design  solutions  (man- rated) ,  or  can  be  obtained  by  machine 
alcne  (automatic);  (3)  that  if  man’s  inclusion  in  the  system 
is  justified  by  his  performance  of  mission-critical  functions, 
his  utilitarian  capadsility  may  be  exploited  for  performemce 
of  other  system  tasks  which  are  not  cost-effective  to  automate; 
and  (4)  that  man  has  csrtain  unique  limitations  which  require 
some  type  of  personnel  support  system  to  accommodate  his 
physical,,  physiological,  and  psychological  constraints  wherever 
he  is  used.  This  philosophy  suggests  that  the  potential  role 
of  man  in  systems  should  be  based  on  four  questions: 

1.  Can  man’s  unique  capabilities  be  significant  in  the 
attainment  of  the  system  goals?  While  it. is  difficult 
to  categorize  man's  unique  capedsilities,  they  seem  to 
lend  themselves  to  two  major  groups,  as  follows: 


■  a.  Man  Ijas  the  ability  to  learn;  that  is,  acquire  new 
knov/ledge  amd  skills,  Man  can  learn  by  practice, 
trial  and  error,  or  transfer  of  previous  training. 
This  unique  ability  to  learn  and  tramsfer  that 
learning  to  another  situation  has  importamt 
implications  for  man’s  potential  role  in  a  system. 
First,  man  cam  perform  in  many  complex  system 
situations  if  those  situations  are  merely  similar  to 
the  learning  situation;  second,  man  can  accomplish 
deliberate  or  insightful  learning  "on-the-job"  if 
the  occasion  calls  for  it. 

b,  Man  has  the  capacity  for  creative  cognition.  This 
unique  capaibility  is  frequently  referred  to  as  the 
ability  to  "think,"  but  man’s  truly  unique  charac¬ 
teristic  is  that  he  is  capable  of  insightful  or 
heuristic  thinking.  On  this  basis,  it  may  be  said 
that  man  is  unique  in  his  ability  to  exercise 
judgment  in  unstructured  situations,  or  to  form 
concepts. 

2.  What  system  performance  could  be  implemented  by  mzui? 

This  question  is  concerned  with  either  operations 

or  maintenance  performance  which  is  basic  or  clearly 
related  to  mission  success,  and  may  usually  be 
restricted  to  primeury  or  critical  performance 
activities. 

3.  If  a  role  for  man  Is  justified  because  of  his  unique 
cap2d7ilitie8  (question  1)  or  primary  performance 
activities  (question  2) ,  what  other  performance  can  b« 


assigned  to  him  to  take  advantage  of  his  utilitarian 
capabilities?  In  other  words,  if  man  is  justified  in 
the  system  for  other  reasons,  then  full  advantage 
should  be  taken  pf  such  things  as  his  flexibility, 
adaptability,  and  motor  skills,  to  perform  tasks 
which  may  be  very  difficult  or  costly  to  automate. 

4.  Will  man's  unique  limitations  constrain  his  use  in  the 
system?  Together,  these  limitations  con?)rise  what  we 
have  been  calling  "compatibility"  impact  areas,  "this 
question  must  consider  both  system  and  individual 
factors  such  as  the  following: 

a.  Man  has  certain  physical  Characteristics  of  size, 
weight,  shape,  strength,  etc. 

b.  Man  has  physiological  needs,  such  as  air,  nourish¬ 
ment,  environmental  protection,  sleep,  comfort,  and 

-^■^eneral  health  maintenemce. 

c.  Man  has  psychological  needs.  His  performance  can 
change  significantly  as  a  result  of  such  attitudinal 
variables  as  motivation,  frustration,  conflict, 
fear,  etc. 

There  are  at  least  two  other  key  issues  which  should  be 
considered  in  any  discussion  about  the  role  of  man.  These  ere 
that  (1)  user  acceptance  factors  are  most  critical  and  will  have 
a  maximum  effect  on  system  effectiveness  as  a  result  of  the 
determination  of  man's  role;  ‘and  (2)  determination  of  man's 
role  also  means  a  determination  of  whether  man  is  local  (in  the 
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immediate  mission  environment)  or  remote  (at  some  geographical  or 
physical  location  away  from  the  immediate  mission  environment) . 


Man's  Role  and  Acceptance  Problems 

The  morale  of  man  is  frequently  lowered  if  he  is  not  func¬ 
tioning  at  what  for  him  is  a  high  skill  level.  Aristotle 
defined  happiness  as  functioning  at  the  highest  level  one  is 
capable  of.  More  recently,  Nissen  (1954)  ,  in  his  paper  on 
motivation,  stated  that  “capacity  is  its  own  motivation." 
Consequently,  if  a  man's  system  role  doe.s  not  permit  hin.  to 
exercise  his  capabilities  or  capacity,  he  will  become  frustrated 
and  lose  motivation  for  performing  his  assigned  and  expected 
system  duties. 

This  problem  of  not  being  permitted  to  maintain  and  improve 
a  skill  capability  can  become  compounded  by  an  expectancy. 
Frequently,  a  man  will  be  led  to  believe  that  he  will  function 
and  learn  at  a  higher  level  than  in  fact  he  will  on  the  job. 

This  false  expectancy,  often  the  result  of  overzealous  recruit¬ 
ment,  will  increase  the  frustration  due  to  non-use  of  complex 
skills. 

Unfortunately,  little  work  has  been  done  on  acceptance 
problems  in  complex  systems.  However,  some  work  performed  by 
Price,-  Smith,  and  Behan  (1964)  on  acceptance  problems  in  auto¬ 
mated  landing  systems  for  aircraft,  and  the  study  of  morale 
problems  in  the  military  unit,  revealed  the  following  three 
general  principles  regarding  acceptance  of  the  system  role  of 
man . 

iJ  Men  are  generally  accepting  of  system  roles  which 
give  them  an  opportunity  to  exercise  and,  therefore, 
maintain  skills  which  they  feel  are  important  to 
maintaining  their  position  in  the  occupational  and 
social  status  system  in  which  they  are  immersed. 


2.  Men  are  generally  accepting  ot  system  roles  which 
permit  them  to  vary  their  procedures  and  the  manner 
of  accomplishing  their  tasks,  on  their  own  initiative. 
Roles  that  fail  to  permit  man  to  vary  his  procedures 
on  his  own  are  generally  labeled  mechanical. 

3.  Men  are  more  accepting  of  roles  which  permit  them  to 
learn.  In  a  recent  study  (unfortunately,  utilizing 
a  sample  of  only  seventeen)  a  correlation  of  +.61 
(statistically  significant  at  the  .01  level  of  con¬ 
fidence)  was  found  between  how  much  the  men  felt  they 
are  learning  on  their  job  and  their  intentions  to 
'reenlist,  (pp.  70-71) 

In  summary,  it  may  be  said  that  men  generally  accept  those 
roles  that  have  more  responsibility,  authority,  and  opportunity 
to  learn.  This  relates  directly  to  the  consideration  of  man's 
role  in  the  system  as  local  or  remote. 


Local  or  Remote  Roles  for  Men  in  Systems 

Military  systems  may  be  frequently  ccnceived  in  terms  of 
geographical  or  physical  location  of  the  performance  units  within 
the  total  system  configuration.  For  the  present  purposes,  local 
refers  to  the  immediate  mission  environment  in  which  the  system 
operates,  and  remote  refers  to  some  geographical  or  physical 
location  away  from  the  immediate  mission  environment.  A  manned 
bomber,  for  instance,  has  man ■ in  a  local  role  in  the  mission 
environment;  while  a  ballistic  missile  system  has  man  in  a  remote 
role  from  the  immediate  mission  environment.  Price,  Smith,  and 
Behan  (1964)  have  also  discussed  this  issue,  particularly  con¬ 
cerning  man's  local  role  in  a  system;  their  reasoning  is  included 
below. 


When  man  is  included  locally  in' a  system,  one  of 
the  usual  reasons  is  to  have  him  available  to  deal  with 
unusual  and  unforeseen  events.  It  is  man's  recognized, 
aptitude  for  reprogramming  or  redesigning  his  role  on 
the  spot  to  deal  with  the  unexpected  that  is  so  valueible, 
because  it  will  increase  system  reliability.  However, 
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this  is  an  aptitude  of  man,  not  a  subsysteia  output 
achieved  at  no  cost  to  the  system  or  system  designers. 
Like  all  required  outputs  it  is  not  free  but  requires 
inputs.  In  order  to  be  able  to  effectively  redesign  his 
role,  in.  unusual  or  eitiergency  situations,  two  precon¬ 
ditions  must  exist,  and  at  this  point  in  the  design  we 
must  determine  if  in  fact  they  will  exist.  These 
preconditions  are  as  follows: 

a.  The  man  must  understand  the  over-all  function 
of  the  system,  and  more  specifically  the  sidi- 
system  he  is  interfacing  with,  his  role  in  it, 
and  how  all  automatic  functions  operate  for 
which  he  might  have  to  provide  total  or  partial 
back  up.  Where  man  is  not  given  adequate 
explanations  as  to  how  functions  other  than 
his  own  are  performed,  particularly  machine 
functions,  he  will  make  up  his  own  explanations, 
as  has  been  pointed  out  by  Firstman  and  Jordan 
(30).  These  explanations  will  more  than  likely 
be  incorrect  and,  therefore,  not  an  effective 
tool  in  an  unforeseen  situation. 

b.  Man  must  be  proficient  at  rapidly  solving  new 
and  unforeseen  problems  in  the  subsystem 
environment.  It  has  been  demonstrated  that 
this  capability  can  be  learned.  Such  capability 
has  been  labeled  learning  how  to  learn ,  or  more 
sim.ply  as  a  learning  set.  However,  this  ability 
can  be  created  and  maintained  only  by  giving  the 
man  the  responsibility  and  freedom  to  continually 
try  out  new  tasks  and  methods.  Obviously  it  is 
not  possible  to  produce  the  capability  in  man 

to  deal  with  unforeseen  events  by  selection, 
traditional  training  methods,  or  job  guides. 

Therefore,  if  man  is  to  be  placed  in  a  system, 
particularly  locally,  in  order  to  increase  system 
reliability  by  having  him  as  an  additive  for.  dealing 
with  unforeseen  events,  it  is  essential  to  give  him 
as  much  responsibility  and  authority  as  is  feasible. 
Maximum  responsibility  and  authority  are  necessary  to 
permit  him  to  develop  a  learning  sei:  “o  that  he  will 
have  the  capability  to  deal  with  unexpected  events. 

(pp.  27-28) 


An  interesting  perspective  cn  the  local  versus  remote  issue 
in  the  role  of  people  and  systems  appeared  in  a  recent  issue 
(April  1978)  of  the  Armed  Forces  Journal.  Because  of  its  brevity, 
the  item  is  included  below  just  as  it  appeared  in  the  magazine. 

DON'T  BE  HYPNOTIZED 
BY  THE  FLASHING  LIGHTS 

A  senior  Army  leader,  who  has  fought  and  won  in  many 
.combat  actions,  raised  a  fundamental  caution  when 
discussing  the  CP  advances  with  AFJ.  His  concern: 
that  commanders  will  become  so  mesmerized  with  the 
fantastic  display  devices  and  torrents  of  information 
slicing  into  their  command  posts  that  they  will  stay 
there  rather  than  get  out  with  the  troops  fighting 
the  battle-  He  says  it's  easy  for  them  to  convince 
themselves  that  the  CP — not  the  fight— is  the  proper 
place  for  them.  That  is  wrong,  he  says. 

He  was  asked  how  to  prevent  the  "CP  syndrom  [sic]." 

He  considers  the  solution  fairly  easy.  Simply  let 
commanders  take  the  command  post  with  them,  by  using 
micro-miniaturized  terminals  and  display  devices. 

Then  they  can  be  in  the  fight,  but  still  tap  the 
resources  of  the  CP. 

The  particular  item  above  is  also  of  interest  from  the  viewpoint 
that  we  are  experiencing  an  extraordinary  growth  in  hardware  and 
computer  technology,  and  this  technology  growth  will  critically 
impact  the  roles  of  men  in  future  systems.-  It  is  therefore  more 
important  than  ever  that  the  human  factor  not  become  the  forgotten 
factor  at  this  phase  of  system  development. 

To  bring  this  discussion  of  the  role  of  man  to  a  close,  it 
seems  reasonable  to  acknowledge  some  support  of  this  concept  from 
other  authors.  Coburn  (1973),  in  his  document  entitled  "Human 
Engineering  Guide  to  Ship  System  Develppment , "  t2dces  note  of  the 
fact  that  the  justification  for  developing  any  new  Navy  system  is 
not  to  produce  hardware  but  rather  to  achieve  some  specific 
operational  capability.  Coburn  further  notes  that  these 
capabilities  are  generally  the  objective  of  the  General 
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Operational  Requirement  (GOR)  and  that  GOR  43,  "Personnel  Logistics," 
discusses  hxunan  engineering.  While  the  GOR  does  not  use  the  term 
"man's  role,"  it  does  imply  a  similar  responsibility  for  human 
factors  engineering  as  indicated  by  the  excerpt  included  below. 

Human  Factors  Engineering.  This  area  is  primarily 
concerned  with  the  implementation  of  human  operator 
considerations  in  the  development,  operations,  and 
maintenance  of  new  and  current  organizations,  weapons, 
end  support  systems.  The  human  operator  is  defined 
in  the  broadest  context,  to  include  system  managers, 
assigned  leaders,  operators,  maintainers,  and  support 
personnel.  The  requirement  for  successful  integration 
of  peppla  insists  that  qualitative  and  quantitative, 
elements  of  normally  functioning  hvanan  capabilities,, 
within  the  constraints  of  people  resource  availedaility , 
be  the  focal  points  around  which  organizations, 
weapons,  and  support  systems  are  designed. 


Air  Force  Regulation  800-15,  Human  Factors  Engineering  and 
Management,  states  that  one  of  the  major  objectives  of  HFE  is 
to  assure  that  "man's  role  in  the  system  is  defined  in  order  to 
optimize  his  performance  in  relation  to  that  specific  system." 


Finally,  Melching  (1968) ,  in  a  paper  titled  "A  Concept  of 
the  Role  of  Man  in  Automated  Systems,"  discusses  the  problem  of 
man's  role  in  highly  avitomaited  Army  air  defense  weapon  systems. 
An  excerpt  of  particular  pertinence  to  the  topic  at  hand  is 
presented  below. 

It  is  suggested  that  man's  role  in  automated  systems, 
although  perhaps  more  subtle  and  more  difficult  to 
assess,  is  comparadJle  in  significance  to  the  other 
factors.  Thus,  for  exeuople,  just  as  the  designer  of 
a  system  cannot  decide  whether  to  automate  a  given 
set  of  functions  until  lie  is  satisfied  that  automation 
is  within  the  state  of  the  art,  so  should  he  not 
attempt  to  assign  functions  to  man  until  he  has 
arrived  at  a  satisfactory  conception  of  the  general 
role  of  man  in  that  system. 
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In  other  words,  to  guide  him  in  this  thinking  and 
planning,  the  designer  of  an  automated  system  needs 
a  clear-cut  conception  of  the  general  role  of  man 
in  such  systems.  Without  this  basic  guidance,  the 
functions  that  he  allocates  to  man  may  reflect  only 
a  sort  of  "fallout"  from  his  attempts  at  automation, 
rather  than  any  careful  premeditation  on  his  part. 

In  short,  the  designer  needs  a  conception  of  what 
man's  role  should  be  before  he  can  decide  what  it 
will  be. 


In  brief  summary,  it  is  the  opinion  of  the  present  authors 
that  determination  of  man*  s  role  during  the  mission  analysis 
phase  of  the  systems  development  is  a  key  human  factors  product 
upon  which  hinges  the  critical  impact  of  human  factors  with 
respect  to  capability,  cost,  and  compatibility. 

The  Allocation  of  System  Functions  to  Man 


In  every  military  system  development  cycle  there  must  be 
decisions  concerning  (1)  if  man  should  be  in  the  system  or  not, 
and  (2)  if  he  is  in  the  system,  what  he  will  do.  If  one  is 
concerned  with  obtaining  optimal  human  performance  in  military 
systems,  then  clearly,  determining  (1)  whether  man  will  have  a 
role ,  and  (2)  if  so,  what  functions  he  will  participate  in  arc 
two  of  the  most  important  decisions  in  a  system  development  cycle 
that  bear  upon  this  concern.  Subsequent  activities  in  system 
development  that  are  concerned  with  task  analysi.s,  selection, 
training,  and  human  engineering  of  interfaces  for  man  in  the 
system  are  consequences  of  the  role  and  allocation  of  function 
decisions  and  cannot  make  up  for  bad  decisions  in  these  two 
areas. 

Despite  the  importance  of  these  two  decision  steps  concerning 
man's  performance  in  the  system  to  be  developed,  there  has  been  a 
general  concern  in  the  literature  and  eunong  human  factors  profes¬ 
sionals  that  allocation  of  function  decisions  is  Inadequate. 

This  is  not  a  new  concern. 


A-10 


In  a  document  entitled  "Factors  Affecting  Degree  of  Automation 
in  Test  and  Checkout  Equipment,"  which,  among  other  things,  reviews 
the  problems  of  allocation  of  fvmction,  Swain  and  Wohl  (1961) 
assert: 

A  rather  stark  conclusion  emerges:  There  is  no 
adequate  systematic  methodology  in  existence  for 
allocating  functions  (in  this  case,  test  and 
checkout  functions) between  man  and  machine. 

This  lack,  in  fact,  is  probably  the  central  problem 
•in  human  factors  engineering  today  ...  It  is 
interesting  to  note  that,  ten  years  of  research 
and  applications  experience  have  failed  to  bring 
us  closer  to  our  goal  than  did  the  landmark  article 
by  Fitts  in  1951  (p.  9). 


In  an  article  entitled  "Allocation  of  Functions  Between 
Man  2md  Machines  in  Automated  Systems,"  Jordan.  (1963)  discusses 
current  problems  and  efforts  to  allocate  functions  between  men 
and  machines,  and  arrives  at  a  similar  conclusion  to  that  of 
Swain  and  Wohl.  Jordan's  final  conclusion  is  stated  as  follows: 
"Herein  lies  the  main  future  challenge  to  h'oman  factors 
engineering."  (p.  165) 


Jordan  also  presents  an  analogy  drawn  from  the  physical 
sciences  and  concerned  with  the  concept  of  "ether,"  which: 

.  .  .  played  a  central  role  in  physical  thinking 
for  over  a  century  after  having  first  been 
introduced  as  a  necessary  medium  for  propagating 
electromagnetic  waves.  But  during  ail  this  time 
all  attempts  to  build  and  expand  upon  this  concept 
led  to  difficulties  and  contradictions.  A  century 
of  research  on  ether  turned  out  to  be  sterile  in 
that  no  significant  advance  was  made  during  that 
time. 


The  conclusion  which  Jordan  draws  from  this  analogy  is  as 
follows: 

The  lesson  to  be  learned  from  this  momentous  episode 
is  that  when  a  scientific  discipline  finds  itself 
in  a  dead  end,  despite  hard  and  diligent  work,  the 
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dead  end  should  probably  not  be  attributed  to  a 
lack  of  knowledqe  of  facts,  but  to  the  use  of  faulty 
concepts  which  do  not  enable  the  discipline  to  order 
the  facts  properly.  Tne  failure  of  human  factor 
engineering  to  advance  in  the  area  of  allocation  of 
functions  seems  to  be  such  a  situation  .... 


During  studies  of  higniy  automated  Army  Air  Defense  Systems, 
Melching  (1968)  arrived  at  a  similar  concern  over  the  allocation 
of  functions  in  systems,  as  indicated  below. 

With  the  aid  of  extremely  capable  electronic  ■ 
computers,  such  systems  are  able  to  process  vast 
amounts  of  envi ronmental  and  other  data.  The 
capability  of  these  systems  is  such  that  they 
can,  at  least  theoretically,  conduct  an  entire 
battle  without  the  assistance  of  man. 

Such  a  prospect  is  awesome,  to  say  the  least. 

As  a  consojuence ,  th<i  builders  and  users  of  such 
systems  have  shown  a  strong  inplination  to  design 
them  so  that  a  manual  override  of  some  sort  is 
possible.  No  one,  it  seems,  is  willing  to  let 
the  machine  make  all  the  decisions. 

Manual  -’rride,  however,  is  only  one  of  several 
issues  .t  tend  to  arise  in  connection  with  highly 
automut  .d  systems.  Numerous  other  questions  also 
appear.  For  example,  in  what  specific  ways  and  in 
what  circumstances  wo\ild  man  be  justified  in  inter¬ 
vening  in  the  actions  and  decisions  of  an  Automated 
system?  Should  man  ever ' be  given  duties  other  than 
that  of  system  override?  Should  man  ever  function 
in  series  with  the  machine  component?  Or  should'  he 
function  only  in  a  parallel  backup  fashion! 

A^  these  questions,  of  coursev  are  expressions  of 
a  problem  that  has  long  plagued  system  designers-^' 
that  of  allocation  of  functions  between  mar  and 
machine. 


Men  and  machines  are  not  competitors.  This  statement  is 
parsunount  when  one  is  concerned  with  the  allocation  of  a  system 
fxinction  for  performance  by  a  man  or  a  machine.  It  is  equally 
poor  system  design  to  have  a  man  doing  a  machine* s  job  or  a 
machine  doing  a  man’s  job. 


Many  system  or  function  performance  requirements  may  be 
implemented  by  solutions  involving  man  as  part  of  the  design 
concept.  Such  solutions,  wherein  it  is  feasible  to  use  man 
as  part  of  the  solution,  are  called  man- rated  {from  Price  & 
Tabachnick,  1968).  Thus,  man- rated  performance  is  any  par- 
formemce  that  can  be  obtained  with  man  as  part  of  the  design 
solution.  The  essential  question  of  function  allocation  is 
what  functions  are  man-rated  and  what  is  the  most  advantageous 
extant  of  mem's  participation. 

The  ramge  of  human  participation  in  potential  man-rated 
solutions  may  actually  be  a  continuum;  however,  for  the  sedce 
of  simplicity  only  the  ends  of  the  continuum  need  be  defined. 

These  two  points  are  simply  called  manual  and  automatic. 

Manual  performance  implies  that  a  man  performs  the  function; 
that  he  generates  or  accomplishes  whatever  power,  energy,  or 
energy  transduction  is  required;  and,  furthermore,  that  he  controls 
the  application  of  power  or  directs  the  utilisation  of  the  given 
energy.  No  assumptions  are  made  about  the  nature  of  the  activity. 
It  may  utilise  human  receptors  or  effectors,  or  both.  The  defi¬ 
nition  does  not  preclude  the  use  of  tools  (e.gf.,  a  chart,  a 
lever,  or  a  telescope)  which  merely  extend  mem's  raw  caped&ilities. 

Automatic  performance  implies  that  a  machine  performs  the 
function  by  generating  or  accoirplishing  whatever  power,  energy, 
or.  energy  transduction  is  required;  and  that  a  man  controls,  in 
real  time,  tiie  application  of  the  power  or  directs  the  utilisar 
tion  of  the  given  energy.  In  automatic  function  performance  man 
participates,  but  indirectly.  He  may  determine  what  is  to  be. 
done,  and  perhaps  how,  as  In  the  use  of  a  digital  or  analog 
computer.  He  usually  monitors  the  output  to  determine  whether 
it  meets  certain  minimal  performance  standards.  He  initiates 
and  may  terminate  the  operation  of  the  automatic  device,  as  in 
the  use  of  an  autopilot. 
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As  may  be  seen  from  the  discussion  above',  the  problem  of 
allocation  of  functions  to  men  in  many  cases  becomes  a  decision 
as  to  the  degree  of  automation.  The  decision  to  automate  a 
function  in  many  cases  makes  the  rcie  of  man  qualitatively  more' 
demanding.  Recenc  advances  in  the  state  of  the  art  in  engineering 
and  computers  have  produced  machines  with  tremendous  capacities 
and  speeds  which  may  require  fewer  personnel  to  operate,  but  which 
also  m.ay  place  increasingly  more  difficult  tasks  on  those  per¬ 
sonnel  who  must  install,  maintain,  monitor,  override,  and. program 
these  systems.  I:  tie  task  complexity  of  personnel  performances 
in  a  highly  automated  system  is  to  be  reduced  to  the  point  where 
highly  skilled  personnel  are  not  required  (or  at  least  reduced} , 
then  the  burden  of  re. ponsibi lity  lies  with  the  human  engineering 
of  the  interface  , between  the  automatic  machine  and  the  humian 
monitor  or  operator.  In  this  way,  training  costs  and  time  for 
personnel  can  be  kept  to  a  minimvOT.  The  general  advantages  of 
the  automated  system  appiy  to  the  extremes  of  any  performaiice 
continuum.  For  example,  monitoring  functions  which  either  do 
not  change  over  long  periods  of  time,  or  change  extremely  rapidly 
in  time,  are  best  performed  automatically.  Those  monitoring 
functions  that  are  Iai  the  middle  of  the  continuum  may  well  fall 
within  the  capabilities  of  a  human  monitor  and  be  performed  by 
man  with  as  'vich  reliability  and  accuracy  as  by  the  machines,  and 
at  much  less  cost. 

Finally,  a  special  consideration  that  needs  to  be  pursued  in 
allccat.ing  functions  is  user  acceptance.  Recent  advances  in 
computer  technology  have  presented  a  temptation  to  system 
designers  to  automate  functions  whenever  possible.  This  may 
create  more  problems  than  it  solves,  particularly  with  respect 
to  the  co.mpatibility  impact  and  user  acceptance. 

Acceptance  problems  could  be  defined  as  any  frustration  of 
any  human  needs.  As  an  example,  excessive  automation,  particularly 
in  information  processing  systems,  may  end  up  overloading  the 
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user  with  more  information  than  he  can  handle.  Excessive  auto¬ 
mation  may  also  restrict  man  from  performing  at  his  highest  skill 
level.  If  automation  does  not  permit  man  to  exercise  his 
capabilities  or  capacity,  he  will  become  frustrated  and  lose 
motivation  for  performing  his  assigned  and  expected  system 
functions.  Problems  created  by  lack  of  confidence  in  the  effective- 
and  reliable  performance  by  hardware  of  automated  functions  must 
be  considered  independently  of  whether  in  fact  the  hardware  is 
effective  and  reliable.  If  man  does  not  accept  a  particular 
automated  fxinction,  he  simply  will  not  perform  in  the  manner  that 
the  system  designer  intended. 

What  is  needed  is  the  development  of  an  awareness  of  the  need 
for  considering  user  attitudes  when  system  design  decisions  are 
being  made.  This  will  permit  the  incorporation  of  acceptance 
factors  as  one  criterion  in  tradeoff  analyses  that  already  include 
a  consideration  of  the  performance  capabilities  and  reliabilities 
of  man  and  equipment  components.  It  may  be  found,  for  example, 
that  a  decision  to  automate  a  particxilar  function  based  upon  sound 
engineering  considerations  would  produce  a  degree  of  negative 
acceptemce  that  would  clearly  offset  the  anticipated  advantages  of 
the  engineering  solution.  Price,  Smith,  and  Behan  (1964) ,  in  the 
study  of  pilot  acceptance  of  automated  landing  systems,  offer  the 
following  principles  as  guidance  concerning  automation  acceptance. 

1.  The  more  system  experience  a  man  has,  with  this 
experience  including  exposure  to  automated 
equipment,  the  more  accepting  he  is  of  the 
automated  equipment  and  the  more  he  will  use 

it  in  the  prescribed  manner. 

I 

2.  Those  with  more  status,  responsibility,  and 
authority  tend  to  be  more  accepting  of  and  make 
more  use  of  automated  equipment  than  others. 

3.  Where  failure  of  the  performance  of  its  function 
by  automated  equipment  can  endanger  the  life  of 
the  man,  he  is  less  likely  to  accept  and  use  it 
despite  prescribed  procedures. 
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4.  There  is  generally  high  acceptance,  within  the 
limits  of  the  above  three  principles,  of  the 
automation  of  servo  tasks,  particularly  those 
which  must  be  performed  over  long  periods  of 
time. 

5.  There  is  generally  rather  low  acceptance  of 
automation  of  decision  maiking  functions. 

In  concluding  this  discussion,  it  is  salient  to  note  that 
the  allocation  of  system  functions  to  man  (or  machine)  is  recog¬ 
nized  as  a  hojnan  factors  product  by  DOD  and  all  three  services. 

DOD  recognizes  the  definition  and  allocation  of  function 
in  MIL-H-46855E ,  Human  Engineering  Requirements  for  Military 
Systems,  Equipment  and  Facilities. 

The  Army  recognizes  function  allocation  in  AR  602-1,  Human 
Factors  Engineering  Prograun,  and  in  HEL  Guide  1-69 ,  Manpower 
Resources  Integration  Guide  for  Army  Material  Development. 

The  Navy  recognizes  function  allocation  in  NAVMATINST 
3900.9,  Human  Factors;  in  the  Human  Engineering  Guide  to  Ship 
System  Development  ^Coburn,  1973);  and  a  report  on  Humam  Factors 
Engineering  for  Navy  Weapon  System  Acquisition  (Baker  et  al., 
1979).  . 


The  Air  Force  recognizes  function  allocation  (num-machine 
analyses)  in  AFR-800-15,  Humam  Factors  Engineering  and  Managemant 
and  in  AFSC  Design  Handbook  1-3,  Human  Factors  Engineering. 


Task  Analysis  and  Determination  of 
Human  Engineering  Requirements 


As  suggested  by  the  title,  this  human  factors  product  has 
two  parts.  The  first  part.  Task  Analysis,  is  a  delineation  of 
the  specific  task  performance  (both  operator  and  maintenance) 
required  to  be  performed  by  man.  The  second  part.  Human  Factors 
Engineering  Requirements,  is  more  concerned  with  how  man  is 
expected  to  accomplish  those  tasks  (at  least  some  of  them  will  be 
aided  by  human  engineering) ,  and  the  identification  of  infor¬ 
mation  and  response  requirements  (interfaces)  between  man  and  the 
system.  Human  factors  personnel  should  provide  (1)  both  the 
methodology  for  and  performance  of  task  analysis,  (2)  the 
identification  of  human  engineering  as  a  means  of  achieving 
ter  assisting)  task  performance,  and  (3)  specific  requirements 
or  techniques  for  man  to  receive  information  from  the  system 
and  to  make  responses  to  the  system. 


Task  analysis  has  been  employed  by  those  individuals  con¬ 
cerned  with  the  "Personnel  Subsystem",  ever  since  people  have  been 
recognized  as  an  integral  part  of  military  systams.  Task  analysis 
as  it  is  generally  practiced  today  was  probably  first  formalized 
by  Miller  (1953! .  It  is  the  basis  for  human  engineering  because 
it  is  necessary  to  know  "what"  is  expected  of  people  in  systems 
before  we  can  p  rescribe  "how"  personnel  are  to  achieve  what  is 
expected  of  them. 

Task  analysis  data  is  clearly  required  by  MIL-H-46855B, 

Human  Engineering  Requirements  for  Military  Systems,  Equipment 
and  Facilities.  A.lso,  the  Tri-Service  Advisory  Group  for  Human 
Factors  is  developing  a  new  task  analysis  requirement. 


The  need  :cr  task  analy=>j.s  to  support  training  and  perfor- 
iT'.ance  itids  har,  also  been  recognized  by  DOD  and  the  services  and 
was  recently  sva^jrart:'!' J  (at  for  maintenance)  by  Foley 

(1978)  in  a  report  on  v.he  impact  of  advanced  maintenance  data  and 
task  oriented  train Lnc  technologies  on  maintenance,  personnel, 
and  training  sysion.o. 

Human  Factors  . r.  ;'h:‘ e  r  t nc  bee:  u i  renen ts 

Achieving  t'e-rsor  iao. _ Per  iormance  .  As  stated  earlier,  once 

it  has  been  uotoiiai-eo  'what"  people  will  do  in  systems  it  is 
necessary  to  ecu  "now"  to  provide  for  achievement  of  the 

expected.  j.e r  f  c  i  rh.;'  to . 

Tiiere  art  ,  ir,  r.  iai,  four  ways  in  which  one  may  develop 
personnel  per  I'c  rm.ar.ce  acnievement : 

i.  Pv.-*r.-;c;"v.  1  s-'Iection 

1.  Training  '  ■ 

3.  Job  aids  and  manuals 

4.  Human  enui nee ring. 

Personnel  selection  technique.?  are  useful  when  a  small 
number  of  personnel  are  required,  highly  specialized  skills  are 
required,  extensi''e  experience  is  required,  system  personnel  are 
to  assist  in  system  uevelopment,  and  the  system  is  essentially  a 
one— shot  attempt. 

Training  is  a  valuable  (but  expensive)  technique  when  all  of 
the  performance  requirements  can  be  specified,  a  relatively  large 
number  of  system  p'er.sonnel  will  be  involved,  system  personnel 
will  be  a  permanent  or  semi- permanent  complement,  skill  require-, 
ments  are  relatively  Jiigh  but  not  specialized,  and  extensive 
experience  is  not  re<;uired. 


Performance  aids  and  manuals  will  always  be  required  in 
military  systems.  However,  performance  aids  are  particularly 
valuable  in  cases  where  there  is  relatively  high  personnel 
turnover,  skill  requirements  are  relatively  low,  task  performance 
can  be  specified  in  detail,  and  large  numbers  of  system  personnel 
are  involved. 

Human  factors  engineering  is  also  a  means  of  achieving  (or 
assisting  in  achieving)  personnel  performance,  as  well  as  reducing 
error  probability.  A  system  which  provides  sufficient  and 
meaningful  information  to  the  human  operator  or  maintainer  emd 
provides  adequate  and  compatible  methods  for  responding  to  system 
demands  c2ui  substitute  for  selection,  training,  and  performance 
aids  in  some  cases.  Furthermore,  human  engineering  becomes  a 
permanent  part  of  the  system  and  the  investment  is  usually  made 
once,  whereas  the  investment  in  training  must  be  made  over  and 
over  as  personnel  turnover  occurs.  Moreover,  human  engineering 
applied  throughout  a  large  complex  system  can  make  training  (and 
cross-training)  easier,  and  can  make  it  easier  for  both  a  novice 
and  an  experienced  individual  to  operate  or  maintain  the  system. 
Thus,  a  significant  human  factors  activity  is  to  determine 
what  types  of  hurneui  performance  can  best  be  achieved  or  assisted 
by  huraeui  engineering. 

No  matter  what  the  primary  method  for  achieving  human  per- 
formzmce  is,  human  engineering  will  affect  the  reliability  with 
which  man  can  perform  his  intended  role,  functions,  and  tasks 
whenever  he  must  interface  with  the  systean.  Data  need  to  be 
available  to  system  designers  with  respect  to  enhancing  human 
reli£d5ility  through  enhancing  both  the  behavioral  and  attitudinal 
interface  between  man  and  the  system--the  conqpatibility  factor. 

Human  Information  and  Response  Requirements.  Finally,  as 
part  of  the  human  factors  engineering  product  a  determination 
should  be  made  of  information  and  response  requirements  necessary 


for  the  user  to  interface  with  the  system.  The  terms  information 
and  response  are  deliberately  used  in  place  of  displays  and 
controls  since  the  actual  selection  or  design  of  displays  and 
controls  should  occur  after  it  is  known  what  information  and 
response  will  be  required  of  the  system  user.  This  will  permit 
consideration  of  combining  information  on  a  sihgle  display  or 
time-sharing  the  display  or  similar  considerations  which  are  only 
possible  after  all  of  the  information  requirements  are  known. 

The  same  thing  is  true  for, response  requirements  and  the  eventual 
design  of  controls. 

The  human  may  receive  information  either  from  the  environ¬ 
ment  directly  or  through  displays  of  the  machine;  he  may  also 
make  responses  directly  to  the  environment  or  to  the  machine 
through  its  controls.  In  an  operating  system,  all  inputs  either 
to  the  man  or  machine  may  be  considered  system  demands.  All  out¬ 
puts  from  the  man  or  machine  may  be  considered  system  performance. 

A  final  observation  should  be  made  that  the  maui  and  machine 
inevitably  operate  in  some  kind  of  environment,  ,  This  could  be  a 
physical  environment,  such  as  the  roadway  or  the  atmosphere;  and 
it  could  be  an  organizational  environment  such  as  an  aircrew, 
communications  network,  or  a  management  information  processing 
function.  No  matter  what  the  context  of  the  environment,  it 
imposes  demands  on  the  man-machine  system,  which  in  turn  responds 
to  provide  system  performance  in  the  operating  environment. 

Design  of  Optimal  Man-Machine  Interface 

An  optimal  man-machine  interface  design  is  that  which  is  the 
most  desirable  from  the  human  factors  viewpoint  while  remaining 
within  the  constraints  of  the  overall  system  design.  This  human 
factors  product  probably  needs  the  least  explanation  and  the 
least  justification  for  being  included  as  part  of  the  system  and 
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equipment  design  decisions  during  the  Full-Scale  Development 
Phase.  It  is  during  this  phase  that  design  decisions  will  result 
in  hardware,  software,  and  procedures  with  which  people  in  systems 
must  interact.  These  design  decisions  and  their  compatibility 
with  the  human  physically,  behaviorally ,  and  attitudinally  are 
the  loci  for  h’jman  errors  in  system  performance.  While  not  all 
human  errors  are  disastrous  with  respect  to  system  performance, 
certainly  no  one  would  argue  that  human  error  must  be.  minimized, 
particularly  when  this  can  be  done  at  minimum  cost  before  hardware 
software,  and  procedures  are  released  for  production.  The  answer 
simply  lies  in  having  avail2d3le  data  and  expertise  which  will  be 
a  part  of  design  decisions  concerning  mem-machine  interfaces  in 
the  Full-Scale  Development  Phase. 

MIL-STD-1472B  is,  of  course,  a  fundamental  requirement  of 
complex  systems  ander  development  and  provides  the  basis  for  the 
design  of  optimal  man-machine  interfaces.  There  are  many  other 
widely  accepted  handbooks  or  guidebooks  which  provide  information, 
primarily  about  man's  physical  and  behavioral  characteristics, 
that  must  be  accounted  for.  However,  even  at  this  level  of 
system, design  it  is  still  essential  to  consider  the  attitudinal 
varied?  les. 

Through  training  and  experience  man  has  built  up  memy  habit 
patterns  that  lead  him  to  expect  things  to  look,  sound,  or  feel 
a  certain  way.  Conversely,  there  is  a  psychological  phenomenon 
known  as  perceptual  constancy  which  allows  us  to  perceive  certain 
things  for  what  they  are,  even  though  they  are  distorted  or 
symbolic..  This  has  relevance  in  the  design  of  the  equipment 
interface,  as  certain  types  of  instrument  symbols  are  more 
acceptedsle  than  others  because  they  meet  man's  perceptual 
expectancy  or  do  not  exceed  his  bounds  of  perceptual  constancy. 

A  practical  exeunple  of  this  is  the  symbolic  representation  of 


the  runways  as  part  of  a  head-up  display — some  representations 
are  simply  more  acceptable  than  others. 

This  completes  the  discussion  of  the  four  principal  products 
of  human  factors  P.SD.  It  is  the  opinion  of  the  authors  that  there 
is  ample  rationale  and  precedent  for  the  establishment  of  these 
products  as  an  integrated  part  of  system  development. 
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APPENDIX  B 

DERIVED  VOCABULARY  LIST  OF  HUMAN  FACTORS  METRICS 

Systtm-RtiatMl  Twins  Msd  Associated  Oimens.ons  (Unit  of  Measure) 


ACCESSIBILiTY 

ACCURACY 

CAPABILITY 

COMPATIBILITY 

CRITICALITY 


$tjDiect‘''e:  satisfactory/ijrisatisfat  toi v  ease  of  ad 
mission  to  various  areas  of  an  irem 

probability/frequenc/  of  documented  error 

subiective:  mission  objective  achievable  given  the 
condition  during  the  mission 

subjective;  ability  of  items  of  equipment  to 
coexist  (including  effect,  of  temperature  and 
moisture 

subjective:  relative  degree  of  task  importance  for 
mission  success 


DURABILITY  probability;  item  will  survive 

a)  its  projected  life 

b)  Overhaul  point 

c)  rebuild  point 

without  a  durability  failure  (failure  that  causes 
an  item  to  be  rebuilt  or  replaced) 


EASE  OF  USE 

FAILURE  RATE/FREQUENCY 


FIRING  RATE 
HABITABILITY 


subjective;  tasks  associated  with  simplicity,  reada¬ 
bility,  etc. 

1)  number  of  failed  items 

2)  number  of  effects  (out  of  tolerances)  per  month, 
week,  hour,  etc. 

time  (measured  from  firing  to  reloading  of  weapon) 

subjeaive;  adequacy/ease  of  space,  transport, 
watch  standing,  rest,  relaxation,  workspace 
and  access 


MALFUNCTION,  SYSTEM  frequency  per  unit  time  (hours)  based  on  avail- 

INITIATEO  able  reliability  data  &  maintenanca  data 

MEAN  FLIGHT  HOURS  .  mean  probable  flight  hours  between  maimeniiKe 

BETWEEN  MAINTENANCE  ,  actions 

ACTION 

MEAN-MAINTENANCE  TIME  1)  mean  hours  preventive  end  corrective  mainte- 

rtanca 

2)  total  preventive  and  corrective  maintenartba 
time  divided  by  total  number  of  preventive  and 
corrective  actions  during  a  specified  interval 

(MTBAMA)  MEANTIME  seme  as  MTBF  except  all  maintenarsce  actions  are 

BETWEEN  ANY  collected  as  data 

MAINTENANCE  ACTION 
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1)  mean  time  a  system  functions  until  occurrence 
of  a  failure  requiros  corrective  maintenance 
(characteristically  over  a  two-rnonth  period) 

2)  total  functioning  life  ot  a  population  of  items 
divided  by  the  total  number  of  failures  within 
the  population  during  a  measurements  cycle 
(time,  cycles,  miles,  events,  etc.) 

(MTBM)  MEAN  TIME  mean  of  the  distrioution  of  time  intervals  between 

BETWEEN  MAINTENANCE  maintenance  actions 

(MTBUMA)  MEAN  TIME  same  as  above  except  only  unscheduled  mainte- 

BETWEEN  UNSCHEDULED  nance  is  collected  as  data 

MAINTENANCE  ACTION 


(MTBF)  MEAN  TIME 
BETWEEN  FAILURE 


(MTTR)  MEAN  1 IME  TO  total  corrective  maintenance  time  divided  by  total 

REPAIR  number  of  correut've  maintenance  actions 

during  a  specified  interval 


(MTTR^)  MEAN  TIME  TO 
REPAIR  (ACTUALLY 
ACHIEVED) 


(MTTPtp)  MEAN  TIME  TO 
REPAIR  (FLIGHTLINE) 


total  correcHve  and  preventive  maintenance 
time  divided  by  total  number  of  corrective 
and  preventive  maintenance  actions  during 
a  specified  interval 

mean  probable  time  spent  in  fligh'dirse  mainte¬ 
nance  before  system  is  returned  to  a  readv- 
for -operation  condition 


(MTTR|)  MEAN  TiME  TO  total  corrective  maintenance  time  divided  by 

REPAIR  (INHERENT)  total  mimber  c(  corrective  maintenance  actions 

during  a  specified  interval 

(MTTRq)  mean  time  to  total  corrective  maintenance  time  divided  by 

REPAIR  (OPERATIONAL)  total  number  of  ccrrecti'/e,  preventive,  ad¬ 

ministrative,  and  suppert  maintenance  actions 
during  a  specified  interval 

(OPERATIONAL)  SUITABILITY  tubjaciive: 

1)  establishment  of  system  operability  in  - 

operational  environment  (withirt  stated 

constraints) 

2)  identification  of  adequate  instrumentation, 
comfort,  visibility,  hailing,  etc.  of  systems 
by  personnel 

(PILOT)  WORKLOAD 

PROOUCIBILITY 


subjective:  degree  of  effort  required  to  accomplish 
a  specific  task 

(TSiE  application):  scbjective  ability  of  dif¬ 
ferences  between  prototype  and  production 
models  to  achieve  desirable  result  (as  a  result 
of  ECP  8i  program  change  orders) 


READY  RATE, OPERATIONAL  %  of  essigrrsd  itents  capable  of  pefforming  an 

jssigrseo  mission  or  fuctetion 


SAFETY 


1)  probability  ot  injury  or  damage 

2)  subjective;  satisfactory/unsatisfactory  materials, 
fire  &  explosion  protection,  mechanical  % 
electrical  hazards) 

SERVICEABILITY  time:  ability  to  service  in  specified  in terv;.l 

STANDARDIZATION/  degree  of  similaiity  (lack  of  ambiguities)  of 

COMMONALITY  OF  DESIGN  two  displays  designed  to  same  specifications 

aitd  standards 

SUBSYSTEM  EFFECTIVENESS  subjective;  the  technical  capability  of  a  sub¬ 
system  (RADAR,  FLIR,  etc..,)  to  accomplish 
a  specific  task 

SURVIVABILITY  probability  that  a  system  will  withstand  hostile 

man-made  environnaent  and  retain  mission 
accomplishment  capability 

TIME,  DOWN  (DOWN  TIME)  time  (hours,  frequency,  duration)  which  an  item 

it  not  in  condition  to  perform  its  specified 
function 

TRANSPORTABILITY  subjective:  ease  of  transit,  packaging,  load/ 

unloading,  sacurity  &  fastening 

WEA  ROUT  rate  of  increase  in  feilure  rate  of  items  over  system 

life  (cycles,  time,  miles) 


Pancnnai-Raiatad  T«rin$  and  Aiao^iatad  Dimantioni  (Unit  of  Maature) 


ACCIDENT  RATE 
ACCURACY 


ANXIETY 

APTITUDE  AND  SKILL 

ATTRITION/TURNOVEH 

DISSATISFACTIO.\S/ 

SATISFACTIONS 

EFFICIENCY 

ERROR  RATE  (ANALYSIS) 


ILLUMINATION  LEVEL 

INJURY 


number  per  specified  number  of  hours 

1)  kill/no  kill  ratio 
21  %  correct 

3)  subiective:  associe'ed  with  cognitive  skills 
(e.g.,  observinc;,  estimating,  detecting,  recog¬ 
nizing,  pcisitioning,  reading,  etc...) 

At  measure  of  precision  and/or  timeliness  of 
performance 

subiective;  stress  factors  associated  with  pilots 
(e.g.,  training,  confidence) 

1)  testing  scores  (e.g.,  AFQT) 

2)  subjective:  low  vs.  high 

%  attrition— number  of  attrited  personnel  divided 
by  number  of  attrited  personnel  plus  number 
of  nonettrited  personnel 

subjective:  ratings  of  challenge,  personnel-job 
mat(^,  perceived  degree  of  utilization 

ratirtg  success  on  a  task 

1 )  mean  error  per  performance  time 

2)  percent  and/or  number  of  operator  error 
(e.g.,  for^tting,  accidents,  inability,  etc...) 

3)  analysis;  includes 

a)  amplitude 

b)  frequency 

c)  type 

d)  change  over  time 

1)  measure:  luminance 

2)  subjective:  number  of  lighting  deficiencies 

subjective:  injury  type,  severity,  frequency 


maintenance,  CORRECTIVE  number,  rate,  frequency  of  acts  performed  to 

restore  an  item  to  a  specified  condition 


MAINTENANCE,  PREVENTIVE 


MALFUNCTION,  HUMAN 
INITIATED 

(MOBAi  MILITARY 
OPERATIONS  IN 
BUILT-UP  AREAS 


number,  rete,  frequency  of  actions  perfornwd 
to  retain  an  item  in  a  specified  condition 

frequency  of  test  participant  (operator)  error 
resultirtg  in  systam/item  malfunction 

1)  communications  distance  (limitations) 

2)  weapons  effectivertess. 

3)  tactics  effactiverteu 
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MORALE 

MOTIVATION 

NIGHT  OPERATIONS 

NOISE/BLAST 

PERFORMANCE  TIME 
OR  RATE 

PRODUCTIVITY 

PROFICIENCY 

RADIATION 


subjective;  ratings  cf  i'idividual  personnel  identifi¬ 
cation  and  satisfaction  with  work  group,  job 
activities,  duties,  supervision,  etc. 

suojective;  rating  of  desire  to  perform  duties, 
obtairi  experience,  advance 

performance  (target  identification*  in  night 
missions 

sourtd  pressure  riiejsurements(e.g..dbs.  amplitude, 
also  iwlocity,  wavelength  frequency  in  herz) 

mean  time/number  per  some  unit/rate 


units  produced  per  some  interval 
test  scores  (written) 

radiation  effects  aircrew  performance  on  radiation 
environments 


REACTION  TIME  1)  (time  reaction);  uptime  to  initiate  a  mission, 

measi<red  from  the  time  the  command  is 
received 

2)  operator  perception  time  (or  start  time)  in 
response  to  some  initiating  stimulus 

STRENGTH  amount  lifted  (kilograms) 

STRESS,  GENERAL  gas  (general  adaptation  syndrome) 

STRESS, TASK  OVERLOAD  subjective;  woikload  excessiveness 


TASK  COMPLEXITY/ 
DIFFICULTY 

TASK  DURATION 

TASK  FREQUENCY 

TEMPERATURE 

TIME,  ADJUSTMENT/ 
CALIBRATION 

TIME,  CHECKOUT 


TIME,  FAULT  CORRECTION 

TIME,  FAULT  (ISOLATION) 
LOCATION 


subjective:  rating  based  on  knowledge  and  skill 
required  for  performance 

total  time  required  for  task  completion  (also  as 
in  tracking  targets>%  of  time  on  target) 

number  of  responses  rgade  by  an  operator(s) 
in  a  specified  interval 

measures  of  comfort  and  performance  in  variable 
temperatures 

time  .required  to  make  needed  response 

time  required  to  verify  performance  of  an  item 
(in  specified  condition) 

time  lequired  to  correct  a  failure 


,  tihie  (hours)  measured  from  discovery  of  a  fault/ 
failure  to  correct  identification  of  failed  item 


TIME,  TASK  TIME 
TIME,  TURNAROUND 

USER  ACCEPTANCE 

VAPORS/EMISSIONS 

VIBRATION 

WINOFORCE  (Q-FORCE) 


time  required  to  perform  task 

time  required  to  serviM  or  check  out  an  item 
for  recommitment 

subjective:  underuse,  misuse,  abuse  of  equipment 
due  to  dissatisfaction  with: 

a)  machine  function 

b)  status 

c)  economic  fears 
d|  survival  fears 

e)  enjoyment  of  manual  performance  of  tasks 

measured  in  parts  per  million  (PPM)  over  specified 
time 

frequency  (in  He)  over  a  unit  exposure  time 

windspeed  indicator  (impact  on  physical  operating 
environment) 

subjective  level  of  effort  requirsd  to  accomplish 
a  task 


WORKLOAD 
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EXAMPLES  OF  HUMAN  FACTORS  EFFORTS  RELATED 
TO  SYSTEM  DEVELOPMENT  PRODUCTS  AND  METRICS 

Application  of  th,e  Metrics  Approach  to  the 
Evaluation  of  Human  Factors  Products 

Previous  chapters  have  demonstrated  that  human  factors 
products  are  explicitly  tied  to  the  formal  sequence  of  military 
system  development,  and  that  the  stages  represent  a  progression 
from  the  general  to  the  specific.  It  should  also  be  remembered 
that  each  human  factors  product  exists  in  order  to  provide  a 
response  to  a  generic  problem  in  system  development. 

The  mode  of  illustration  in  this  section  will  be  to  fill  in 
some  of  the  cells  of  the  matrix  in  Exhibit  C-1  with  specific 
metrics.  The  intent  is  to  demonstrate  that  these  assignments 
can  be  made  in  a  sensible  manner.  In  several  cases,  we  will 
also  go  a  further  step  and  show  how  empirical  measures  fit  into 
the  metric  level.  Again,  the  objective  is  demonstration,  not  an 
exhaustive  explication. 

Supplementary  Evaluation  Considerations 

An  important  aspect  in  the  evaluation  of  the'  human  factors 
contribution  to  military  system  development  should  be  the  quality 
of  the  human  factors  products;  that  is  their  "intrinsic"  merits. 
An  example  is  a  human  factors  report  that  is  assessed  pn  its  , 
overall  relevance  or  cogency,  the  logi.c  of  the  derivation  of  its 
conclusions,  the  validity  of  the  data  used  in  the  inferential 
reasoning  presented,  and  even  its  readability. 

■Our  more  structured  approach  is  not,  intended  to  preclude 
such  a  mode  of  assessment.'  Indeed,  that  kind  of  evaluation 
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Kxhibit  (M 

Uliutradoru  of  Metrics  Used  to  Link  iiu^nan  Factors  Products  and  Impact  Areaa 


is  virtually  spontaneous.  V.-e  would  expect  ..1  human  factors 
products  to  he  evaluated  in  teat  morp  judcp,.ental  mode.  Such 
judgmental  evaluaticTis  ar^  useful  and  needed  but  they  do  not  go 
far  enough.  Specifically,  a  report  such  as  is  mentioned  above 
could  be  entirely  cogent  and  valid  and  still  not  be  valuable  to 
the  system  designers.  Certainly  they  would  not  perceive  it  as 
valuable  unless  the  recommendacious  were  seen  to  mal'.e  a  positive 
difference  in  the  cost,  capability  or  compatibility  of  the 
system.  To  do  so  requires  the  linkage  'with  their  concepts  of 
system  impact  areas — "caste,"  in  their,  (the  designer's  and  system 
development  manager's)  terminology. 

Metrics  and  Principal  Human  Factors  Products 

Human  Role  x  Cost.  We  have  filled- in  the  cell  in  the  upper 
left  hand  corner  of  the  matrix  with  the  metric  of  cost  associated 
with  depot  maintenance,  as  shown  in  Exhibit  C-1.  The  rationale 
for  this  example  is  that  the  hximan  role  in  the  maintenance 
process  is  implicitly  or  explicitly  defined  when  the  system 
maintenance  philosophy  is  promulgated  early  in  the  design  prodess, 

A  typical  parameter  at  this  level  is  the  threshold  of  the  choice 
of  repairing  a  subassembly  at  the  operational  unit  or  replacing 
it.  If  the  philosophy  is  to  replace  most  subassemblies,  the 
function  of  repair  and  the  cost  associated  with  that  repair, work 
becomes  a  depot  responsibility.  The  human  factor  aspect  would 
be  the  extent  to  which  quick  and  accurate  fault  diagnosis  could 
be  performed  at  the  operations  unit  level.  That  outcome,  in 
turn,  would  be  influenced  by  the  quality  of  available  test 
equipment  and  job  aids  for  operational  unit  level  maintenance 
technicians. 

Given  some  actual  data  on  these  parameters  plus  a  consideration 
of  the  complexity  of  the  system,  spare  parts  availability,  etc., 
and  an  accurate  characterization  of  the  fault  diagnosis  performance 
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of  maintenance  technicians  in  similar  situations,  the  human 
factors  recommendation  might  be  to  work  toward  the  circuit-card 
level  of  replacement  as  opposed  to  the  subassembly  level.  If 
such  a  recommenL.ation  were  valid,  it  could  lower  maintenance 
costs  and  could  link  across  to  such  metrics  as  system  availability 
in  the  sense  that  time- to-repair  could  be  reduced. 

Allocation  of  Functions  x  Cost.  An  example  is  shown  in 
which  the  metric  cost  associated  with  a  design  modification  to  a 
radio  device  is  cast  in  terms  of  allocated  functions  between  men 
and  machine  (represented  by  the  cell  at  the  intersection  of  the 
Allocation  of  Function  Col'omn  and  the  Cost  row)  .  Modern  technolpgy 
is  not  prone  to  simple  devices  and  a  "black  box”  fix  can  often 
add  to,  rather  chan  alleviate  user  problems.  For  example,  the 
automatic  tuner  for  the  AN/GRC-19 — a  high  frequency,  AM,  medium 
power  radio — turned  out  to  be.  more  trouble  than  it  was  worth. 
Development  commenced  in  the  early  years  of  World  War  II  and  the 
set  v/as  fielded  in  1949.  Prior  to  the  AN/GRC-19,  standard  radio 
sets  required  the  operator  physically  to  change  the  length  of 
the  antennae  and  to  go  through  a  series  of  "dipping"  and  "peaking" 
operations  to  tune  the  transmitter  to  the  operating  frequency. 

A  well-skilled  operator  was  required  to  get  the  most  out  of  the 
radio,  and  training  such  operators  was ' not  easy.  The  designers 
of  the  .^N/GRC-19  sought  to  eliminate  this  training  problem  by 
incorporating  a  "black  box",  an  automatic  tuning  assembly  which 
was  expected  to  substantially  decrease  requirements  for  operator 
training,  as  well  as  i.'icreasing  the  speed  and  accuracy  of  tuning. 

The  AN/GRCrl9  passed  its  acceptance  tests  and  was  pat  into  use. 

But  over  the  years  (last  procurement  was  1965) ,  the  Army  experienced 
a  net  loss  in  system  cost-effectiveness.  While  the  transmitter 
tuned  more  rapidly,  the  operator  needed  as  much  training  to 
manipulate  the  "black  box"  a.-:  the  former  manual  system  demanded, 
arid  the  training  of  maintenance  personnel  had  to  be  increased  to 


take  care  of  the  tuner.  The  following  data  reveal  the  cost  and 
other  associated  effects  upon  the  cost-effectiveness  of.  having 
an  automatic  tuner  added  to  the  AN/GRC-19  radio. 


Cost  +25% 

Size  and  Weight  +15% 

Repair  Costs  +10% 

Speed  of  Tuning  Improved 

■  Operator  Training  Same 

Maintainer  Training  Increased 


In  sum,  the  Army  bought  rapid  tuning  at  a  substantial  price  in 
higher  procurement,  shipping,  maintenance  and  training  costs. 

Had  the  cost-effectiveness  of  the  machine— allocated  function 
been  compared  with  the  original  man-allocated  version  (as  was 
done  retrospectively  in  the  data  summarized  above),  the  designers 
would  have  seen  no  utility  in  the  design  modification.  (Portions 
adapted  from  TRADOC  Pam  71-8) 

Task  Analysis  and  Human  Engineering  Requirements  x  Cost. 

Cost  savings  are  anticipated  through  increased  development  and 
use  of  simulators  for  training  and  skills  maintenance,  especially 
in  the  areas  of  maintenance  and  flight  training.  Simulators 
have  been  touted  as  capable  of  reducing  or  eliminating  a  need 
for  operational  systems  and/or  spare  parts,  since  all  necessary 
functions  (e.g.,  malfunctions)  afe  simulated.  In  addition,  with 
increasing  costs  associated  with  flight  training  and  air  skills 
maintenance  due  to  energy  resource  consumption,  simulators  offer 
an  economical  alternative  means  for  skill  development  and  retention 
Especially  desircdsle  is  the  maintenance  of  combat  readiness  for 
pilots  in  Air  Combat  Maneuvering  (ACM),.  The  practice  required 
to  reach  optimal  readiness  levels  places  high  cost  factors  upon 
actual  aircraft  and  fuel  resources.  Use  of  simulators  offers  an 


opportunity  to  escape  heavy  costs  and  achieve  desirable  readiness 
states.  Maintenance  trainers  such  as  that  developed  for  the 
Heads-up  Display  tester  associated  with  the  A-6E  aircraft  offer 
an  attractive  $700,000  savings  per  copy. 

In  order  tc  safeguard  huge  investments  in  simulator  R&D, 
much  human  factors  (HF)  R&D  has  been  invested  to:  1)  determine 
characteristics  relevant  to  critical  performance  in  the  opera¬ 
tional  environment  which  must  be  simulated  (through  task  analysis 
and  human  engineering),  and  2)  verify  the  effectiveness  of 
simulators  developed  from  such  research.  While  the  advent  of  the 
age  of  simulators  presents  HF  R&D  with  a  unique  opportunity  for 
empirical  simulator  research,  the  cost-effectiveness  of  such 
research  must  be  demonstrated  through  actual  improvements  in 
skill  proficiency  and  readiness..  While  cost  savings  postulated 
through  a  one-for-one  substitution  of,  simulators  for  actual 
systems  results  in  an  obvious  numerical  cost  figure,  no  accounting 
is  made  of  training  and  readiness  losses  or  gains  made  through 
the  substitution.  Perhaps  more  called  for  is  a  detailed  cost- 
effectiveness  modeling  approach  which  takes  into  account  such 
benefits  as:  reduced  training  time,  train.ing  experience  with 
rare  or  hard-to-duplicate  events  or  contingencies,  better 
monitoring  of  student  performance  by  instructors,  etc.  It  is 
understood  that  recognition  gained  through  simulator  usage  will 
of  necessity  be  shared  with  the  simulator  design  and  training 
communities,  but  nonetheless  human  factors  claims  a  co-equal 
share  of  the  responsibilities  and  benefits.  Cost  savings 
demonstrated  throggh  the  use  of  simulators  will  to  a  great  extent 
vindicate  human,  factors  input  and  investment  in  this  technology. 

Man-Machine  Interface  Design  x  Cost.  Cost  savings  demonstrated 
through  an  increase  in  the  nvimber  of  personnel  made  available  to 
operate  a  system  can  be  a  powerful  metric  for  demonstrating  the 
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underlying  value  of  the  H.F  R&D  that  led  to  the  savings.  A 
computer  modeling  procedure,  the  crewstation  assessment  of  reach 
model,  was  developed  to  simulate  operators  in  aircraft  cockpits 
so  as  to  determine,  among  other  things,  the  percentage  of  the 
pilot  population  which  could  safely  operate  controls  as  well  as 
be  safely  accommodated  by  cockpit  size  and  arrangements.  This 
model  is  used  to  "step— through”  various  cockpit  design  configura¬ 
tions,  with  the  intent  of  evaluating  the  man-machine  match, 
before  the  aircraft,  and  its  cockpit  are  actually  built.  Use  of 
this  model  has  increased  the  percentage  of  aviators  available  for 
such  aircraft  as  the  F-18.  This  has  resulted  in  the  achievement 
of  subst^  . -ial  cost  savings  in  terms  of  manpower  alone,  variously 
described  as  being  between  $5  and  $40  million  a  year.  In 
addition,  .'ther  long  terra  cost  savings  may  be  achieved,  such  i 
those  associated  with  reduction  in  aircraft  accidents,  possible 
redesign/retrofit  of  aircraft  cockpit  configurations,  and  the 
availability  of  a  new  design  evaluation  tool  for  use  by  industry 
on  a  continuous  basis.  Whatever  the  actual  cost  savings  achieved 
(and  a  detailed  cost- benefit  model  may  more  fully  exploit  this)  , 
the  cost  of  the  HF  R&D  has  been  fully  exceeded  by  received 
benefits  to  the  system  as  well  as  reduced  costs. 


Human  Role  x  Compatibility.  Had  human  motivation  been 
considered  a  vital  metric  in  measures  of  system  performance  at 
the  time  vhen  large  multiple  man-machine  systems  (e.g.,  SAGE) 
first  evolved  from  manual  to  automated  operations,  user  acceptance 
would  have  readily  been  seen  as  a  primary  component  of  motivation 
leading  to  mission  effectiveness.  User  acceptance  by  implication 
is  heavily  compounded  with  job  satisfaction.  For  jobs  to  be 
satisfying  three  conditions  seem  to  be  necessary:  the  system 
must  demand  the  operator  to  use  skills;  the  job  must  be  meaning¬ 
ful;  and  the  operator  mast  perceive  real  responsibility  in  job 
performance.  In  designing  and  thinking  about  our  new  con^lex 
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II 


automated  man-machine  systems  we  must  learn  to  design  for  men 
jobs  that  are  intrinsically  interesting  and  satisfying. 


A  researcher  who  v;as  involved  with  the  SAGE  progrcim  makes 
the ' following  comparisons: 


One  notices  a  striking  difference  when  comparing  the 
behavior  of  the  crews  in  the  old  manual  Air  Defense 
Command  sites  to  the  crews  in  the  SAGE  direction  centres. 
In  the  manual  site  almost  every  crew  member  took  pride 
in  his  job.  I  had  occasion  to  visit  many  of  them,  and 
in  every  site  the  crew  members  to  whom  I  talked  would 
eagerly  go  to  great  trouble  to  explain  to  me  the, 
intricacies  of  their  job  and  what  it  demanded  of  them 
for  good  performance.  A  comparable  pride  and  eagerness 
was  almost  completely  lacking  in  the  SAGE  direction 
centres  I  visited.  Men  just  cannot  be  proud  of  something 
which  bores  them. 

If  we  look  closely  at  the  job  demands  in  a  SAGE 
direction  centre,  we  find  several  striking  differences 
between  it  and  the  manual  sites  it  replaced.  First, 
for  most  jobs,  skill  requirements  have  been  reduced 
to  a  bare  minimum.  Second,  most  of  the  jobs  have 
become  so  isolated  and  fractionated  that  they  have 
become  meaningless  in  terms  of  the  overall  craw 
mission  responsibility.  One  clear-cut  example  of 
this  isolation  and  fractionation  will  here  be  given; 
there  are  many  others. 

Most  of  the  jobs  in  an  air  defense  jsystem  involve 
relaying  information;  i.e.,  information  is  processed 
or  acted  upon  and  then  relayed  to  another  position 
for  further  processing  and  action.  Each  job  by 
itself,  although  clearly  defined,  generally  heis 
little  meaning  when  the  total  pictrre  of  crew's 
actions  a:e  lacking.  In  the  manual  sites  there  was 
a  central  plotting  board  which  shoved  such  a  picture 
for  all  the  crew  members  to  see.  Mo  such  summarizing 
display  is  available  for  a  member  of  a  SAGE  crew. 

Third  and  last,  because  of  the  fantastic  performance 
ability  of  the  computer,  because  of  the  inflexibility 
of  even  the  roost  so-called  flexible  program,  and 
because  of  the  mystery,  to  the  crew  members,  of  what 
goes  on  inside  the  computer,  and  reinforced  by  the 
effect  of  the  preceding  two  conditions,  the  roles  of 
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the  human  operator  in  SAGE  and  the  computer  have 
functionally  been  reversed.  Rather  than  the  machine 
being  an  aid  to  the  man,  the  man  becomes  an  aid  to  the 
machine.  In  addition  to  boredom  generated  by  the 
reduction  of  skill,  there  is  a  feeling  of  futility 
generated  by  the  feeling  of  having  lost  control  over 
what  is  going  on.  Maybe  this  is  all  we  desire  of  the  , 
men  in  our  emerging  complex  automated  man-machine 
systems,  that  they  merely  be  aids  to  the  machine,  but 
it  is  legitimate  to  raise  the  question  whether  this 
desire  is  itself  desireUale. 

in  designing  complex  systems,  regardless  of  our  good 
intentions,  we  can  often  create  a  situation  that  becomes 
intolerable  for  the  humam  being,  and  as  a  result  he 
either  leaves  the  system  or,  if  he  cannot,  he  subordinates 
himself  to  the  system  and  ceases  to  play  the  role  which  is 
the  ultimate  role  of  men  in  man-machine  systems,  to  see 
to  it  that  the  eyatem  works. 

(from  Jordan,  1968) . 


As  was  made  explicit  in  the  preceding  discussion,  not  only 
should  the  Role  of  Man  be  considered  early  in  system  development 
to  include  elements  of  motivation,  personnel  satisfaction,  and 
user  acceptance;  but  by  direct  extension  a  case  can  be  made  for 
systematic  development  of  measures  such  as  user  acceptance  and 
metrics  such  as  motivation  to  document  HF  R&D  improvements  in 
systems.  Improvements  in  user  acceptcuice  not  only  benefit  man 
as  a  user,  but  must  also  ultimately  pay  off  in  improvements  to 
overall  system  performance  as  well.  This  may  especially  be  the 
case  for  emergency  and  contingency  situations  in  automated 
systems  (e.g.,  SAGE). 

Allocation  of  Function  x  Compatibility.  The  metric  chosen 
to  demonstrate  the  linkage  in  this  case  is  operadsility.  The 
allocation  "problem"  is  dramatically  illustrated  by  the  case  of 
automated  (computerized)  landing  systems.  Such  systems  may  not 
be  operable  because  they  violate  human-  factors  principles  with 
respect  to  user  acceptance.  Specifically,  it  has  been  feasible 
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for  several  years  to  control  carrier  landings  by  -ters  used 

in  conjunction  with  seme  advanced  radar  telemetry  devices.  In 
spite  of  their  demonstrable  accuracy,  such  systems  are  not  cost- 
effective  because  the  pilots  disengage  or  under-use  them. 

All  such  systems  have  a  manual  override  feature,  for  obvious 
reasons.  Pilots  exercise  the  override  feature  even  when  the 
system  is  working  perfectly  because  they  cannot  bring  themselves 
to  invest  their  trust  in  a  system  in  which  a  "slight"  malfunction 
could  cost  their  life  or  career.  From  the  pilot's  point  of 
view,  the  advantage  is  not  worth  the  potential  cost/risk  associated 
with  a  malfunction--particuiarly  if  the  system  might  be  susceptible 
to  nonobtrusive  malfunctions. 

Test  data  could  show  that  the  probability  of  a  mishap  or  a 
missed  approach  is  significantly  reduced  by  the  automatic  system. 
Thus,  from  an  engineering  viewpoint  the  pilot  is  wrong.  However, 
from  an  outside  functional  point  of  view,  the  system  is  inoperable 
and  might  as  well  not  be  on  board.' 

The  empirical  measure  in  this  case  could  have  been  a  survey 
of  pilot  attitudes.  Had  this  been  done  and  a  human  factors 
report  been  produced  when  the  allocation-.of- function  decisions 
were  being  made,  a  system  might  have  been  designed,  that  would 
have  been  compatible  with  pilot  attitudes,  operable,  and  thus 
effective.  '  Note  that  this  case  also  .links  back  to  cost  consider¬ 
ations.  In  effect,  all  the  development  costs  of  this  system  were 
lost  when  it  becaune  apparent  that  the  system  was  inoperable. 

Task  Analysis  and  H\iman  Engineering  Requirements 
X  Compatibility.  The  metrics  chosen  to  demonstrate  the  linkage 
between  task  analysi.s  euid  human  engineering  requirements  and 
compatibility  are:  performance  and  effectiveness.  The  Army  has 
shown  compatibility  factors  (e.g.,  crew  turbulence)  which  have 
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clear  effects  upon  mission  success,  armor  weapon  accuracy,  speed 
of  use,  as  wal 1  as  overall  crew  cohesiveness.  Identification  of 
pertinent  design  factors  at  the  task  analytic  and  human  engi¬ 
neering  requirements  levels  would  serve  to  mitigate  some  of  the 
user  problems  seen  in  the  field.  The  U.S.  Army  has  long  been 
concerned  witn  getting  the  maximum  capabilities  and  effectiveness 
out  of  its  armor  weapon  systems.  Much  of  the  capability  of  any 
weapon  system  is  a  function  of  the  performance  of  the  crewmen 
assigned.  Some  people  in  the  armor  community  have  expressed 
concern  that  crew  turbulence — the  movement  of  crewmen  from  crew 
to  crew  and  position  to  position- -may  have  a  negative  impact  on 
tank  system  effectiveness.  Research  conducted  during  recent 
years  has  addressed  this  notion  and  attempted  to  identify  the 
relationship  between  tank  crew  turbulence  and  tank  crew  performance 

I 

Tank  crews  contain  four  crewmen,  a  tank  commander — commonly 
called  a  "TC,"— a  gunner,  a  driver,  and  a  loader.  For  the  tank 
weapon  system  to  achieve  full  potential,  each  must  perform 
effectively  in  his  assignee!  position.  Each  duty  position  within 
the  tank  system  requires  unique  skills  and  smooth  coordination 
with  the  other  crew  members.  The  TC  must  identify  and  range  on 
targets,  communreate  his  findings  to  the  gunner  and  loader,  and 
be  prepared  to  guide  the  driver  through  difficult  terrain  based 
solely. on  voice  commands.  The  gunner's  response  to  the  TC's 
identification  of  a  target  must  be  epordinated  with  the  loader's 
response  to  the  TC's  comjnand  specifying  the  type  of  ammunition 
to  be  loaded.  The  accurate  synchronization  of  these  duties  is 
essential. 


Three  types  of  turbulence  were  identified.  They  were: 
equipment,  personnel,  and  position  turbulence.  Equipment  turbun 
lence  occurs  when  a  crew  is  moved  from  one  tank  to  another. 
Personnel  turbulence  occurs  when  crewmen  are  moved  from  one  crew 


to  another,  but  kept  in  their  positions.  And  position  turbulence 
occurs  when  crewmen  are  moved  from  one  position  to  another. 
Assignment  changes  which  create  personnel  euid  position  turbulence 
are  always  accompanied  by  equipment  turbulence.  From  the  data, 
it  appeared  that  position  turbulence  had  a  significant  degrading 
effect  on  gunnery  performeuice.  However,  for  equipment  and 
personnel  turbulence,  little  or  no  effect  was  indicated.  All 
types  of  turbulence  could  be  minimized  if  it  were  possible  to 
assign  each  crewman  to  a  permanent  position,  tank,  and  crew  upon 
his  arrival  in  the  unit.  However,  this  ideal  procedure  is  often 
not  feasible,  because  a  sufficient  number  of  trained  TC  and 
gtinner  replacements  are  not  always  available  to  fill  vacated 
positions.  Consequently,  units  must  fill  TC  and  gunner  positions 
from  available  crewman.  To  cope  with  the  turbulence  required  by 
the  assignment  system,  a  unit  may  frequently  move  crew  members 
up  within  crews,  where  possible,  or  between  crews  where  necessary. 
These  problems  with  crew  turbulence  which  have  direct  effects  on 
metrics  of  performance  cind  effectiveness  are  am  example  of 
compatibility  issues  which  need  to  be  brought  before  equipment 
design  engineers.  Human  factors  personnel  need  to  identify 
design  recommend atioiis  which  achieve  desiradsle  levels  of  standard¬ 
ization  across  crews'  positions  in  order  to  facilitate  (among 
ether  things)  cross- training  and  thus  reduce  the  negative  aspects 
of  crew  turbulence. 

(Portions  from  Eaton  amd  Black,  1980). 

Man-Machine  Interface  x  Compatibi lity.  To  demonstrate  this 
linkage  we  have  chosen  a  particularly  challenging  example:  the 
evaluation  of  the  product  of  detailed  design  by  the  impact  area  of 
compatibility  using  the  producibility  metric.  This  assignment  is 
again  illustrated  in  Exhibit  C-1.  The  challenge  can  be  met  by  the 
consideration  of  the  human  factors  aspect  of  the  production 
process  itself.  What  we  arc  saying  hers  is  that  the  detailed 


design  must  be  compatible  with  the  attributes  of  the  workers  who 
will  fabricate  the  system.  Of  all  the  possible  linkages  between 
human  factors  product,  metric,  and  impact  area,  this  is  one  of 
the  more  likely  to  get  "swept  under  the  rug"  in  the  develtoment 
process.  However,  the  logic  is  not  that  complicated.  For 
example,  at  the  design  recommendation  level  in  the  layout  of  an 
instrument  panel,  the  procedure  can  involve  a  design  review 
against  standard  industrial  fadsrication  practices  that  asks  the 
question:  Is  there  any  aspect  of  the  fabrication  of  this  panel 
that  will  require  deviation  from  standard  practices?  If  the 
answer  is  yes,  the  next  question  is:  Does  the  nonstandard 
requirement  generate  a  possible  mismatch  between  what  roust  be 
accomplished  and  the  physiological,  behavioral,  or  attitudinal 
attributes  of  the  production  workers?  Specifically,  are  parts 
involved  that  are  so  small  that  positioning  is  difficult  for 
individuals  with  normal  vision  ^md  normal  dexterity?  Does 
fabrication  invol/e  the  assembly  of  pieces  by  touch  because  the 
worker  cannot  observe  the  back  of  the  panel  after  a  particular 
production  stage  has  been  reached? 

In  short,  the  human  factors  products  at  the  detailed  design 
level  should  incorporate  not  only  a  recommendation  that  will 
lead  to  an  effective  interface  between  the.  human  operator  of  the 
system  and  the  control  psmel,  but  should  also  include  a  consider¬ 
ation  of  ttie  compatibility  of  the  design  with  the  tools, 
practices,  and  attributes  of  the  humans  who  will  put  the  control 
pane)  together. 

I 

As  indicated,  the  main  impact  can  be  designated  as  compati¬ 
bility  but,  again,  it  should  be  aotad  that  a  good  design  in  this 
instance  would  have  effects  in  the  cost  area,  and  probably  other 
metrics  as  well. 


Human  Role  x  Capability.  Automated  Battlefield  Systems  are 
being  developed  at  a  rate  which  may  surpass  the  ability  of  HF  R&D 
to  offer  input  to  each  in  sufficient  quantity.  The  Army  alone  is 
procuring  approximately  60  new  automated  •  itical  systems  in  the 
coming  decade.  Role-of-man  decisions — m  lue  explicitly  or  implicitly 
by  design  engineers — will  have  a  great  influence  not  only  on  human 
performance  capcibilities  and  limitations  but  •»iso  on  overall 
system  and  mission  capability.  To  make  matte:  :;  more  critical, 
automated  systems  have  had  a  history  of  problems  related  to  man’s 
role,  complexity,  as  well  as  system  hardware  and  software 
architecture. 

As  much  as  comprehensive  HF  R&D  is  required  to  take  advantage 
of  opportunities  offered  by  the  use  of  automated  technology,  with 
the  problems  just  mentioned,  a  method  to  assess  the  impact  of  human 
factors  on  the  course  of  development  for  these  systems  is  also 
necessary.  For  excunple,  systems-embedded  training  within 
fielded  tactical  systems  offers  a. rare  opportunity  for  training 
operators  to  readiness  states  required  for  successful  mission 
completion  in  actual  combat  environments.  The  opportunity  for 
HF  R&D  to  aid  in  the  refinement  of  embedded  training  is  limited 
only  by  the  resources  available  to  fund  it.  HF  R&D  may  be 
invested  in;  software  developments  for  "canned"  scenarios  and 
training  packages,  guaranteeing  the  realistic  nature  of  the 
training  such  as  to  mimic  the  expected  operational  environment 
by  displaying  representative  data  to  system  operation;  as  well 
as  other  areas  of  concern.  Measures  of  personnel  emd  system 
readiness  as  well  as  proficiency,  productivity  (in  terms  of  task 
data  utilized) ,  and  performance  time  or  rate  may  be  used  to 
evaluate  the  effectiveness  of  the  embedded  training  to  success¬ 
fully  train  operators,  and  to  evaluate  skills  acquisition  and 
maintenance  by  operators  undergoing  training;  they  also  provide 
the  capability  to  verify  the  value  of  HF  RSD  that  was  invested  in 
the  effort.  This  component  may  then  be  agg^regated  with  others  to 


form  metrics  such  as  readiness,  performance,  etc.  Finally, 
through  an  analysis  of  type  found  in  the  model  described  in 
Chapter  5,  metrics  may  be  merged  to  assess  the  impact  upon 
syste«Ti  capability.  Given  that  the  information  was  available  on 
the  example  cited,  systems-emJaedded  training,  as  veil  as  other 
relevant  data  (e.g.,  compatibility  of  the  man-machine  integra¬ 
tion,  feelings  of  lack  of  responsibility  and  detachment  from 
system  performance,  effectiveness  of  manual  override  as  well  as 
others  for  each  alternative  configuration  of  a  system),  a  strong 
case  could  be  demonstrated  for  the  value  of  HF  R&D  in  battlefield 
information  systems  development. 

Allocation  of  Functions  x  CapaUaility.  That  the  allocation 
of  functions  between  men  and  machines  is  dependent  upon  the  role 
assigned  man  is  nowhere  as  clearly  demonstrated  as  when  man  is 
remotely  located  in  the  system.  Remote  location  of  man  in  the 
system  eliminates  direct  observation  and  control  of  system 
functions.  Also,  by  implication,  remote  system  monitoring 
requires  heavy  dependence  upon  automation  and  communication 
links.  Needless  to  say,  manual  override  is  totally  surrendered 
to  the  system,  at  least  as  regards  that  portion  from  which  the 
man  is  remotely  located. 

Since  military  intelligence  requirements  consist,  at  least  in 
part,  of  locating  enemy  troops,  and  identifying  concentrations 
and  types  (infantry,  armor,  vehicle,  etc.),  the  military  saw  fit 
to  meet  this  requirement  by  engaging  in  development  of  Remotely 
Piloted  Vehicles  (RPVs) .  RPVs  function  to  scout  the  ene-tiy  in  the 
battlefield,  gathering  intelligence  data  about  them  through  the 
use  of  drone,  aircraft  (the  RPV)  that  cari'y  video  sensors.  Of 
course,  by  design  the  h’iman  is  remotely  located  in  the  system. 

Once  the  location  of  man  in  the  system  is  designated  to  be  remote, 
men's  role  would  be  limited  to  only  jthose  posaibillties  available 
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under  such  circumstances.  Decisions  regarding  the  allocation  of 
functions  to  men  and  machine  will  ca  based  on  this  human  role 
decision,  with  all  its  advantages  and  disadvantages.  Therefore, 
the  mission  success  of  the  RPV  surveillance  depends  on  the  ability 
of  human  operators  to  (a)  detect  significant  information  from 
video  displays,  (b)  ignore  "noise'*  or  non-significant  video 
display  information,  and  (c)  distinguish  friendly  from  enemy 
forces.  It  is  up  to  HF  R&D  at  this  point  to  ensure  positive 
effects  upon  system  capability. 

Allocation  of  functions  within  human  performance  capabilities 
and  limitations  becomes  a  critical  concern  of  system  developers. 
Caro  must  be  shown  in  designating  man-rated  functions  which  are 
within  available  human  performance  capabilities  and  skill  levels, 
as  well  as  other  more  basic  concerns  such  aS  perception  and 
decision  making.  Measures  of  accuracy-in-vigilance  type  tasks 
of  the  sort  to  be  encountered  in  actual  operation  of  the  system 
would  be  useful  in  determining  human  performance  capabilities, 
as  well  as  tas--  ■  rplexity/difficulty.  HF  R&D  may  also  aid’  in 
selecting  aptitu..-.  and  skill  requirements  for  RPV  operators 
through  measures  sucfi  as  proficienc%‘. 

These  .measures,  along  with  others  which  represent  quantifiable 
measures  of  capability,  may  be  aggregated  together  in  a  model 
such  as  that  presented  in  Chapter  6  to  form  metrics  such  as 
general  skills,  performance,  and  task/workload.  These  metrics 
may  then  be  utilized  to  deteemine  the  impact  on  capability  as  a 
result  of  this  HP  R&D  investment. 

Task  Analysis  and  Human  Engineering  Requirements 
X  Capability.  This  cell  is  filled-in  in  Exhibit  C-1  with  the 
metric  "maintainability."  The  mode  of  approach  is  to,  show  that 
the  product  in  this  case  can  be  evaiuaced  by  mea«»>’iing  that 
product  against  maintainability  criteria  that  link  to  the 
capability  of  the  system. 
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Design- for-maintainability  is  a  well  established  concept  in 
the  development  of  military  systems.  In  this  particular  instance, 
we  can  look  at  something  as  simple  as  the  design  of  access 
hatches  for  re-arming  combat  aircraft.  From  an  aerodynamics 
point  of  view,  such  hatches  must  be  flush  and  faired  so  that  the 
fasteners  and  handles  do  not  spoil  the  air  flow  in  flight. 
Consequently,  there  may  be  a  temptation  on  the  part  of  the 
designer  to  overlook  the  fapt  that  the  speed  with  which  these 
hatches  can  be  removed  could  be  crucial  to  the  combat  effectiveness 
of  the  system.  Similarly,  it  can  be  optimal  from  a  human  factors 
point  of  view  that  removal  and  replacement  be  accomplished  using 
conventional  tools,  and  that  the  geometry  of  the  fasteners  and 
the  complete  hatch  be  such  that  it  precludes  errors  such  as 
mispositioping  the  hatch  upon  replacement  or  not  properly 
locking  the  fasteners. 

The  product,  in  this  case,  would  be  a  hatch  cover  design 
that  would  meet  both  the  requirement  for  good  aerodyncunics  and 
the  requirements  for  quick  and  easy  access  and  error-free  hatch  ' 
cover  replacement.  The  evaluation  at  the  empirical  level  would 
be  speed  and  accuracy  of  performance  during  re- arm  operations  in 
a  prototype  test  situation.  A  more  comprehensive  evaluation 
would  be  one  that  would  reveal  the  consequences  of  reduced 
turnaround  time  On  the  overall  mission  effectiveness  of  the 
system.  This  level  of  outcome  can,  indeed,  be  estimated  by 
impact  analysis  techniques  discussed  in  Chapter  6. 

Man-Machine  Interface  Design  x  Capability.  Increased 
accuracy  in  airborne  weapons  use  has  been  a  continuous  goal  of 
researchers  for  obvious  reasons.  As  in  so  many  instances,  weapons' 
accuracy  is  highly  dependent  upon  a  compatible  man-machine 
integration.  One  effort  has  been  to  improve. the  target  acquisition 
side  of  weapon  systems  (as  opposed  to  the  purely  engineering  side, 
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which  involves  just  machine  capability) .  The  primary  element  of 
target  acquisition  of  necessity  involves  a  human  performance 
component.  That  component  has  become  more  critical  in  today's 
airborne  threat  environment#  which  has  increased  in  lethality  as 
well  as  requiring  aircraft  to  be  upgraded  in  continuously  high 
flight  regimes  (speed)  at  the  edge  of  the  aircraft's  flight 
envelope.  This  has  resulted  in.  a  military  requirement  to  aid  the 
human  operator  in  target  acquisition.  Addressing  the  human  com¬ 
ponent  in  target  acquisition  has  resulted  in  refinement  of 
television  (TV)  and  forward-looking  infrared  (FLIR)  systems. 

These  systems  were  designed  to  improve  identification  and  recog¬ 
nition  of  potential  targets,  actual  target  acquisition  and, 
finally,  the  probability  of  achieving  a  kill.  A  critical  element 
in  the  viability  of  such  systems  is  human  performance  capabilities 
as;^ociated  with  perception.  KF  R&O  was  required  in  order  to 
determine  the  density  of  scan  lines  necessary  for  optimal  target 
acquisition  performance  in  one  such  system.  This  R&D  contributed 
to  the  overall  capability  potential  of  the  weapon  system  in  the 
following  areas: 

•  Increased  weapon  accuracy  resulting  in  higher  kill 
ratios 

•  Reduced  acquisition  time 

-  correlated  with  increased  probability  of  achieving 
a  kill 

reduced  vulnerability.. 

Taken  as  measures  of  performance,  these  eureas  may  be  aggregated 
into  a  performance  metric  which  may  also  contribute  to  effective¬ 
ness  and  system  survivability.  A  model  tailored  to  evaluate  the 
contribution  of  this  HF  R&D  to  capability  added  to  target 
acquisition  performance  and  the  overall  system  could  clearly 
demonstrate  that  the  HF  R&O  was  instrumenta.1  to  this  system 
development  effort. 


Summary 


As  shown  above,  it  is  possible  to  identify  a  set  of  metrics 
that  are  demonstrably  acceptable  to  design  engineers  (because 
they  are  extracted  from  engineering  documentation) ,  and  that 
also  serve  to  relate  the  empirical  and  analytical  measures  used 
by  human  factors  specialists.  This  circumstance  encourages  the 
view  that  the  human  factors  products  that  enter  (or  should 
enter)  into  the  military  system  development  process  can  be 
evaluated  in  strictly  engineering  terms. 

Several  cases  were  described  that  tend  to  verify  that  view. 
What  is  still  lacking  is  a  comprehensive  and  orderly  methodology 
for  collecting  the  appropriate  data  and  generating  explicit, 
quantitative  conclusions  about  the  specific  value  of  a  particular 
human  factors  product  for  a  particular  military  system  development 
effort. 
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I  USa  aIR  DEFENSE  B3ARU  ATTN:  FTLES  REPOSITORY 
1 ' OSA  infantry  boars  ATTn:  AT^B^ItfAE 
1  USAnNTELLIGEMCE  CEN  ANU  SCM  aHNI  aTSI.UOTU^SF  - 
1  OSA  uRONANCF  cen  ano  SCh  ATTN|  atsl-tu-tac 
1  USA  aRMOR  school  ATTNI  AUK-Td 

1  USA  aHMUH  CENTEh  UIRECIUHaTE  oF  CUMBAT  UEVCLURMEnTS 
I  NAVAl  postgraduate  sch  AIINI  uUULEY  ANOX  library  (COOE  IAEA! 

,1  USA  IRANSPORTaT ION  school  ULPuTY  asst.  COMMANUANT  EOUCA*  TtCHIvOLOOT 
1  USA  .'.IGNAL  SChUuL  ANU  FT.  bORUON  ATTNI  ATZH'ET 

1  USA  aRmoR  cENTLR  .  FT.  ANUX  OFFICE  OF.  ARMOR  FORCE  MOT  ♦  STANOAROIZATION 
1  CMIE»>  OF  naval  tUjCATION  ANQ  TNG  / 

I  USA  SIGNAL  SCHOOL  •  FT.  GUROOn  EUUCATIOnaL  TECHNOLOGY  DIVISION 
I  HU  AlC/XPrii  training  stsilhs  development 
I  iiSAisu  aTtn;  atsie-oT 
I  US  aaMy  armoh  center  aTTN:  aT/k«Tu.pmo 

1  USA  wUARTtPMASltR  SCHOOL  uIREcTORaTE  OF  TRaININB  OEVci  OPMCNTS 
I  US  Ci.AST  Gl'ARO  ACaUCHV  / 

I  USA  iRAnSPORTAIIOn  SCHUUL  OIRECIORATC  OF  TRAINING  *  OqCTRINC 
I  USA  infantry  SChU3L  LIBRARY  / 

I  USA  infantry  school  AfTNl  ATSH'I'V 
I  US  AwMr  INFANTRY  SCHOOL  ATTNl  AISH-CO 
t  i.'Sa  iNFANfRY  SCHOOL  ATTNI  ATSh-UOT.lRO 
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1  USA  infantry  SCriUOL  ATTN;  AlSH-tV 

1  USA  mP  ♦  CHEM  SCH/TNO  CtN  *  FT.  MCCLEcLAN  ATTN;  aTZN.rTS 

I  USA  mP  ♦  CmEM  Scm/Tno  C£N  ♦ft.  MCCLECLan  OIKi  COMBAT  DEVELOPMENT 

I  USA  mP  ♦  CHEM  ScH/lN(j  CEN  ♦FT.  MCCLELLAN  DIM*  TruIN;  |6  DEVELOPMENT 

I  USa  hP  ♦  LhEM  SCH/TNO  CtN  ♦  FT.  MCCLELLAN  ATTNj  aTZN.mP-ACE 

)  USa  institute  of  AUHiNisiHAiiON  attn:  resident  Training  Management 
)  USa  ♦ield  aRTillerv  School  murrIS  snett  library 
I  USA  iNSTITlTt  OF  ADmIn 1 STHAT lUN  ACADEMIC  lIBRART 
1  USA  aAR  college  aTTN:  LlbRAKY 

I  USA  rNGlNEFR  SChUOL  LIhRAHY  AnU  LEARvInq  RESOURCES  CENTER 

I  USA  armor  school  (usahh:>)  aiin>  library 
I  organizational  CFFbCTlYbNtSS  CEN  ♦  SCH  aTTN:  LIBRARIAN 
I  US  AmMV  INTELLiGb'ICC  CENTER  ♦  SCHOOL  ATTNt  aTSI-TU 

1  US  ammy  Intelligence  ctNitR  ♦  sc:tool  attn:  atsi-rm-m 

I  us  AaMT  INTELUgENCE  center  ♦  SCHOOL  ATTN;  aTS1-TO>LO 

I  US  AaMV  intelligence  CENTER  ♦  SCHOOL  ATTNt  AT$1-CD-CS-C 

1  US  AmMV  JNIELLIGENCE  CENTeR  ♦  SCHOOL  ATTN:  ATSI-oT“Sf-IM 

I  OEPAmTMENT  of  The  aIR  force  air  UNIVEHSIIY  LIBRARY  (aTCI 

1  HQ  TmAOOC  training  UtVELUPMENT  ANSTUUTf 
?  BRITISH  EMhASSY  BRITISH  uEFENCL  STaF^ 

2  CAnAiiIAN  ioiNT  staff 

1  COLS  <M)  Library  . 

1  FRENt.H  army  attache 

1  ausTmIan  Embassy  defense*  miliiart  anu  air  atiaChE 

3  CANADIAN  defence  LUISON  SIaFF  aTTN:  counsellor*  defence  R  and  0 
1  ROYAi  NETHERLANDS  EMBASSY  MILITARY  ATTACHE 

1  CANADIAN  FORCES  base  CDRNMALLIS  ATTNi  PERSONNEL  SCLECTrON 
7  CANADIAN  FORCES  PERSONNEL  APPL  NSCH  u^IT 
I  ARMY  personnel  research  ESTABLISMMtNT 
I  NETHERLANDS  EMBASSY  OFFICE  OF  THE  AlH  ATTACHE 
6  LlBRaRT  OF  CONGRESS  EaCHAnGE  «nO  GIFT  01 V 
I  UEFEmSE  technical  InFOHMAIION  CEN  ATTNi  OT1C*OOA.2 
UO  LIbRaRV  OF  congress  unit  documents  EAPEOITING  project 
1  US  gdvernment  Printing  ofc  lihRary*  public  oocumcmts  department 
l  us  GuVERNMENT  printing  ofc  LIhRAHT  aNU  STaTUTORT,  LlB  Olv  «SLL) 

I  THE  aRMr  LIBRAMy  ATTNt  AHMT  SfUOlES  SEC 

3  ROYAI  ARMY  EUUCaTIONAL  COMPS  CENTRE  ARMY  SCHOOL  Of  TRAINING  SUPPORT 
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